Optimizing Performance in Azure Cosmos DB: Best Practices and Tips

When we are working with a database, optimization is crucial and key in terms of application performance and efficiency. Likewise, in Azure Cosmos DB, optimization is crucial for maximizing efficiency, minimizing costs, and ensuring that your application scales effectively. Below are some of the best practices with coding examples to optimize performance in Azure Cosmos DB.

1. Selection of Right Partition Key

Choosing an appropriate partition key is vital for distributed databases like Cosmos DB. A good partition key ensures that data is evenly distributed across partitions, reducing hot spots and improving performance.

The selection of a partition key is simple but very important at design time in Azure Cosmos DB. Once we select the partition key, it isn't possible to change it in place.

Best Practice

• Select a partition key with high cardinality (many unique values).
• Ensure it distributes reads and writes evenly.
• Keep related data together to minimize cross-partition queries.

Example: Creating a Container With an Optimal Partition Key

var database = await cosmosClient.CreateDatabaseIfNotExistsAsync("YourDatabase");
var containerProperties = new ContainerProperties
{
    Id = "myContainer",
    PartitionKeyPath = "/customerId"  // Partition key selected to ensure balanced distribution
}; 

// Create the container with 400 RU/s provisioned throughput
var container = await database.CreateContainerIfNotExistsAsync(containerProperties, throughput: 400);

2. Properly Use Indexing

In Azure Cosmos DB, indexes are applied to all properties by default, which can be beneficial but may result in increased storage and RU/s costs. To enhance query performance and minimize expenses, consider customizing the indexing policy. Cosmos DB supports three types of indexes: Range Indexes, Spatial Indexes, and Composite Indexes. Use the proper type of wisely.

Best Practice

• Exclude unnecessary fields from indexing.
• Use composite indexes for multi-field queries.

Example: Custom Indexing Policy

{
    "indexingPolicy": {
        "automatic": true,
        "indexingMode": "consistent",  // Can use 'none' or 'lazy' to reduce write costs
        "includedPaths": [
            {
                "path": "/orderDate/?",  // Only index specific fields like orderDate
                "indexes": [
                    {
                        "kind": "Range",
                        "dataType": "Number"
                    }
                ]
            }
        ],
        "excludedPaths": [
            {
                "path": "/largeDataField/*"  // Exclude large fields not used in queries
            }
        ]
    }
}

Example: Adding a Composite Index for Optimized Querying

{
    "indexingPolicy": {
        "compositeIndexes": [
            [
                { "path": "/lastName", "order": "ascending" },
                { "path": "/firstName", "order": "ascending" }
            ]
        ]
    }
}

You can read more about Indexing types here.

3. Optimize Queries

Efficient querying is crucial for minimizing request units (RU/s) and improving performance in Azure Cosmos DB. The RU/s cost depends on the query's complexity and size. 

Utilizing bulk executors can further reduce costs by decreasing the RUs consumed per operation. This optimization helps manage RU usage effectively and lowers your overall Cosmos DB expenses.

Best Practice

• Use SELECT queries in limited amounts, retrieve only necessary properties.
• Avoid cross-partition queries by providing the partition key in your query.
• Use filters on indexed fields to reduce query costs.

Example: Fetch Customer Record

var query = new QueryDefinition("SELECT c.firstName, c.lastName FROM Customers c WHERE c.customerId = @customerId")
    .WithParameter("@customerId", "12345");

var iterator = container.GetItemQueryIterator<Customer>(query, requestOptions: new QueryRequestOptions
{
    PartitionKey = new PartitionKey("12345")  // Provide partition key to avoid cross-partition query
});

while (iterator.HasMoreResults)
{
    var response = await iterator.ReadNextAsync();
    foreach (var customer in response)
    {
        Console.WriteLine($"{customer.firstName} {customer.lastName}");
    }
}

4. Consistency Levels Tuning

The consistency levels define specific operational modes designed to meet speed-related guarantees. There are five consistency levels (Strong, Bounded Staleness, Session, Consistent Prefix, and Eventual) available in Cosmos DB. Each consistency level impacts latency, availability, and throughput.

Best Practice

• Use Session consistency for most scenarios to balance performance and data consistency.
• Strong consistency guarantees data consistency but increases RU/s and latency.

Example: Setting Consistency Level

var cosmosClient = new CosmosClient(
    "<account-endpoint>",
    "<account-key>",
    new CosmosClientOptions
    {
        // Set consistency to "Session" for balanced performance
		ConsistencyLevel = ConsistencyLevel.Session      
});

Read more about the consistency level here.

5. Use Provisioned Throughput (RU/s) and Auto-Scale Wisely

Provisioning throughput is a key factor in achieving both cost efficiency and optimal performance in Azure Cosmos DB. The service enables you to configure throughput in two ways:

• Fixed RU/s: A predefined, constant level of Request Units per second (RU/s), suitable for workloads with consistent performance demands.
• Auto-Scale: A dynamic option that automatically adjusts the throughput based on workload fluctuations, providing scalability while avoiding overprovisioning during periods of low activity.

Choosing the appropriate throughput model helps balance performance needs with cost management effectively.

Best Practice

• For predictable workloads, provision throughput manually.
• Use auto-scale for unpredictable or bursty workloads.

Example: Provisioning Throughput With Auto-Scale

var throughputProperties = ThroughputProperties.CreateAutoscaleThroughput(maxThroughput: 4000);  // Autoscale up to 4000 RU/s 
var container = await database.CreateContainerIfNotExistsAsync(new ContainerProperties
{
	Id = "autoscaleContainer",
	PartitionKeyPath = "/userId"
}, throughputProperties);

Example: Manually Setting Fixed RU/s for Stable Workloads 

var container = await database.CreateContainerIfNotExistsAsync(new ContainerProperties
{
    Id = "manualThroughputContainer",
    PartitionKeyPath = "/departmentId"
}, throughput: 1000);  // Fixed 1000 RU/s

6. Leverage Change Feed for Efficient Real-Time Processing

The change feed allows for real-time, event-driven processing by automatically capturing changes in the database, eliminating the need for polling. This reduces query overhead and enhances efficiency.

Best Practice

• Use change feed for scenarios where real-time data changes need to be processed (e.g., real-time analytics, notifications, alerts).

Example: Reading From the Change Feed

var iterator = container.GetChangeFeedIterator<YourDataModel>(
ChangeFeedStartFrom.Beginning(),
ChangeFeedMode.Incremental);
while (iterator.HasMoreResults)
{
    var changes = await iterator.ReadNextAsync();
    foreach (var change in changes)
    {
        Console.WriteLine($"Detected change: {change.Id}");
        // Process the change (e.g., trigger event, update cache)
    }
}

7. Utilization of Time-to-Live (TTL) for Automatic Data Expiration

If you have data that is only relevant for a limited time, such as logs or session data, enabling Time-to-Live (TTL) in Azure Cosmos DB can help manage storage costs. TTL automatically deletes expired data after the specified retention period, eliminating the need for manual data cleanup. This approach not only reduces the amount of stored data but also ensures that your database is optimized for cost-efficiency by removing obsolete or unnecessary information.

Best Practice

• Set TTL for containers where data should expire automatically to reduce storage costs.

Example: Setting Time-to-Live (TTL) for Expiring Data

{
    "id": "sessionDataContainer",
    "partitionKey": { "paths": ["/sessionId"] },
    "defaultTtl": 3600  // 1 hour (3600 seconds)
}

In Cosmos DB, the maximum Time-to-Live (TTL) value that can be set is 365 days (1 year). This means that data can be automatically deleted after it expires within a year of creation or last modification, depending on how you configure TTL.

8. Avoid Cross-Partition Queries

Cross-partition queries can significantly increase RU/s and latency. To avoid this:

Best Practice

• Always include partition key in your queries.
• Design your partition strategy to minimize cross-partition access.

Example: Querying With Partition Key to Avoid Cross-Partition Query

var query = new QueryDefinition("SELECT * FROM Orders o WHERE o.customerId = @customerId")
    .WithParameter("@customerId", "12345"); 

var resultSetIterator = container.GetItemQueryIterator<Order>(query, requestOptions: new QueryRequestOptions
{
    PartitionKey = new PartitionKey("12345")
});

while (resultSetIterator.HasMoreResults)
{
    var response = await resultSetIterator.ReadNextAsync();
    foreach (var order in response)
    {
        Console.WriteLine($"Order ID: {order.Id}");
    }
}

Conclusion

These tips are very effective during development. By implementing an effective partitioning strategy, customizing indexing policies, optimizing queries, adjusting consistency levels, and selecting the appropriate throughput provisioning models, you can greatly improve the performance and efficiency of your Azure Cosmos DB deployment. These optimizations not only enhance scalability but also help in managing costs while providing a high-performance database experience.