DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Cosmos DB Disaster Recovery: Multi-Region Write Pitfalls and How to Evade Them
  • Evolution of Cloud Services for MCP/A2A Protocols in AI Agents
  • How to Maximize the Azure Cosmos DB Availability
  • Optimizing Performance in Azure Cosmos DB: Best Practices and Tips

Trending

  • Blue Skies Ahead: An AI Case Study on LLM Use for a Graph Theory Related Application
  • Artificial Intelligence, Real Consequences: Balancing Good vs Evil AI [Infographic]
  • Immutable Secrets Management: A Zero-Trust Approach to Sensitive Data in Containers
  • From Zero to Production: Best Practices for Scaling LLMs in the Enterprise
  1. DZone
  2. Data Engineering
  3. Databases
  4. Cosmos DB as Key Object Store

Cosmos DB as Key Object Store

In this article, readers will gain hands-on experience with Cosmos DB as a potential key object store.

By 
Srivatsan Balakrishnan user avatar
Srivatsan Balakrishnan
·
Feb. 05, 23 · Tutorial
Likes (1)
Comment
Save
Tweet
Share
3.5K Views

Join the DZone community and get the full member experience.

Join For Free

 Cosmos DB can be a good candidate for a key-value store. Cosmos DB is a multimodal database in Azure that supports schema-less storage. By default, Cosmos DB containers tend to index all the fields of a document uploaded. We can limit the index properties only to id and partitionkey to make the container a pure key-value store. Prohibiting other fields to be indexed increases performance and reduces RU to query point records. (Cosmos DB cost is measured in RU.) For key object storage, RU tends to be less, but it still depends on the payload size. Performance tests conducted on 100 partitions and 100K records gave a P95 between 25-30ms for read and write.

Let's dive in and implement Cosmos DB as a pure key-value store in C#.

Prerequisites:

  1. Have an account in the Azure portal
  2. Provision Cosmos DB with a SQL account.

There are a number of ways to hook up the cache into a .NET app. Let's define a generic distributed cache interface with some extensions.

C#
 
interface IDistributedObjectCache
    {
        Task<bool> SetObjectAsync<T>(string key, T value, CancellationToken cancellationToken = default);
        Task<bool> SetObjectAsync<T>(string key, string pk1, T value, CancellationToken cancellationToken = default);

        Task<T> GetObjectAsync<T>(string key, CancellationToken cancellationToken = default);
        Task<T> GetObjectAsync<T>(string key, string pk1, CancellationToken cancellationToken = default);
    }


Compared to IDistributedCache from the .NET framework, IDistributedObjectCache gives the extensibility to add a partitionkey as per application needs.

Let's predefine all the connection parameters needed for the CosmosClient connection

C#
 
public class DbConstants
    {
        public const string ConnectionString = "<CosmosDb connection string>";

        public const string CacheDatabaseId = "ObjectCacheDb";

        public const string CacheContainerId = "CacheContainer";
    }


Let's define a CosmosClientManager which helps us to manage the CosmosClient connection object. You can change ApplicationRegion to your desired location.

C#
 
public class CosmosClientManager : IDisposable
    {
        public CosmosClientManager()
        {
            if (!string.IsNullOrWhiteSpace(DbConstants.ConnectionString))
            {
                CosmosClient = new CosmosClient(DbConstants.ConnectionString, new CosmosClientOptions
                {
                    ConnectionMode = ConnectionMode.Direct,
                    IdleTcpConnectionTimeout = TimeSpan.FromMinutes(30),
                    ApplicationRegion = Regions.WestUS2,
                });
            }
        }

        public CosmosClient? CosmosClient { get; } = null;

        public void Dispose()
        {
            CosmosClient?.Dispose();
        }
    }


Let's define an object model on how records in a cache container will look like

C#
 
public class ObjectContainer<TPayload>
    {
        [Required]
        [JsonProperty("id")]
        public string PrimaryKey { get; set; }

        [Required]
        public string PartitionKey { get; set; }

        public TPayload Payload { get; set; }

        [Required]
        public string ContainerId => DbConstants.CacheContainerId;

        public string PartitionKeyPath => $"/{nameof(PartitionKey)}";

        [JsonProperty("_etag")]
        public string ETag { get; }

        [JsonProperty("_ts")]
        public long DateTimestamp { get; set; }
    }


I created a simple IWarmup to initialize components in a background thread.

C#
 
internal interface IWarmUp
    {
        Task<bool> WarmUpAsync();
    }


Let's define the core CosmosDistributedObjectStore that implements all needed interfaces.

C#
 
public class CosmosDistributedObjectStore : IDistributedObjectCache, IWarmUp
    {
        private readonly CosmosClientManager cosmosClientManager;

        public CosmosDistributedObjectStore(
            CosmosClientManager cosmosClientManager
            )
        {
            this.cosmosClientManager = cosmosClientManager;
        }

        public Task<T> GetObjectAsync<T>(string key, CancellationToken cancellationToken = default)
        {
            cancellationToken.ThrowIfCancellationRequested();

            return GetObjectAsync<T>(key, key, cancellationToken);
        }

        public async Task<T> GetObjectAsync<T>(string key, string pk1, CancellationToken cancellationToken = default)
        {
            cancellationToken.ThrowIfCancellationRequested();

            var container = GetCurrentContainer();

            if (container == null) return default;

            var record = await container.ReadItemAsync<ObjectContainer<T>>(key, new PartitionKey(pk1), cancellationToken: cancellationToken);

            return record != null ? record.Resource.Payload : default;
        }

        public Task<bool> SetObjectAsync<T>(string key, T value, CancellationToken cancellationToken = default)
        {
            cancellationToken.ThrowIfCancellationRequested();

            return SetObjectAsync<T>(key, key, value, cancellationToken);

        }

        public async Task<bool> SetObjectAsync<T>(string key, string pk1, T value, CancellationToken cancellationToken = default)
        {
            cancellationToken.ThrowIfCancellationRequested();

            var container = GetCurrentContainer();

            if (container == null) return false;

            var record = new ObjectContainer<T>
            {
                PartitionKey = key,
                PrimaryKey = key,
                Payload = value,
            };

            await container.CreateItemAsync(record);

            return true;

        }

        public async Task<bool> WarmUpAsync()
        {
            var entry = new ObjectContainer<dynamic>();
            var containerProperties = new ContainerProperties(entry.ContainerId, entry.PartitionKeyPath);

            // optimize container as a pure keyvalue store. Disable indexding other than id column
            containerProperties.IndexingPolicy.ExcludedPaths.Add(new ExcludedPath() { Path = "/*" });

            var db = await cosmosClientManager.CosmosClient?.CreateDatabaseIfNotExistsAsync(DbConstants.CacheDatabaseId, 1000);
            var container = await db?.Database?.CreateContainerIfNotExistsAsync(containerProperties, 1000);

            return true;
        }

        private Container? GetCurrentContainer()
        {
            return cosmosClientManager?.CosmosClient?.GetContainer(DbConstants.CacheDatabaseId, DbConstants.CacheContainerId);
        }
    }


I would like to emphasize WarmUpAsync() and how to fine-tune container creation to the index id and primaryKey fields only in order to make the container work as a pure Key Object Store.

            containerProperties.IndexingPolicy.ExcludedPaths.Add(new ExcludedPath() { Path = "/*" }); 

Let's write a simple caller to test our new key object store.

C#
 
static async Task Main(string[] args)
    {
        var cosmosDistObjectStore = new CosmosDistributedObjectStore(new CosmosClientManager());


        IWarmUp components = cosmosDistObjectStore;
        IDistributedObjectCache objectCache = cosmosDistObjectStore;

        await components.WarmUpAsync();

        var studentKey = "alpha";
        var studentObject = new { Name = "alpha", Age = 10, Class = "Grade-V" };

        // set cache
        await objectCache.SetObjectAsync(studentKey, studentObject);

        // get student grade details
        var alphaDetails = await objectCache.GetObjectAsync<dynamic>(studentKey, studentObject.Class);

        // get all cache
        var alpha = await objectCache.GetObjectAsync<dynamic>(studentKey);
    }


We now have a robust alternate NoSQL key object store with Azure Cosmos DB that can partition and scale on demand.

Cosmos DB Key-value database Performance improvement

Opinions expressed by DZone contributors are their own.

Related

  • Cosmos DB Disaster Recovery: Multi-Region Write Pitfalls and How to Evade Them
  • Evolution of Cloud Services for MCP/A2A Protocols in AI Agents
  • How to Maximize the Azure Cosmos DB Availability
  • Optimizing Performance in Azure Cosmos DB: Best Practices and Tips

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!