Using Autonumber in Couchbase

During the software development process, we often come across a situation where we need to generate a unique key (of an entity) in an orderly sequential fashion (either increasing or decreasing order). Common examples include:

1. Storing entries of a log file with an auto-generated sequence number assigned to each row of data
2. Storing business entities in a database and having a primary key generated from an incremental sequence number.

In the relational database world, this is achieved by making use of something known as “database sequence.” The sequence is a feature provided by most database products which simply creates unique number values in an orderly manner. It just increments a value and returns it. Without a database sequence, it is not very easy to generate unique numbers in a particular order. That’s why it is a popular choice when it comes to populating a primary key (or any unique key for that matter) with unique auto increasing values.

Other ways to generate randomly unique keys include using features such as GUID or UUID. However, there is no guarantee regarding the auto-increment nature that you get when using a database sequence generator.

No Sequence in NoSQL Databases

Unlike the relational database world, there is no straightforward built-in sequence generation feature available for most of the NoSQL databases on the market. One could argue that it’s not usual for a distributed system with free-form data to have a unique auto-incrementing number serve as a unique key on a document because of the conflicting numbers generated in the case of cross-replication of data among different nodes and shards. Instead, implementing a UUID seems a much more viable option to guarantee uniqueness. However, if you need a randomly generated unique ID in a sequence then you somehow need to have an auto incrementing sequence column in a NoSQL database, because the UUID solution would not preserve the sequencing nature of the generated numbers. The main question: How would you handle that using Couchbase? Well, Couchbase has you covered and that’s what this post will describe.

Set up a Couchbase Bucket to Save the Data

Let’s say we are storing items from a product catalog into Couchbase, and while storing the products in a Couchbase bucket we need to set a generated sequence in each of the product data JSON so that it can be used as a uniquely identifiable “key” of a document.

To do so, follow the steps below to create a bucket named “prodcat.”

• First, log into the administrative console of Couchbase.
• Type http://localhost:8091/ui/index.html in your browser.
• Login with admin username and password.
• Navigate to the Data Buckets tab and click on Create New Data Bucket.

• Leave all the other fields as default and click on “Create.”
• Once the bucket is created successfully it will be listed with 0 items. 

Using Counter Documents

Couchbase handles sequence generation by what is known as a “counter” document. Counter is a document that can be incremented or decremented sequentially. An important thing to note here is that the increment or decrement operation of the counter is atomic. When we insert a business entity (such as Product in our case) as a JSON document, we can use the counter document with a key pattern to generate a sequence.

The following code snippet initializes a counter document with an initial value of 20.

// create a connection to couchbase cluster and bucket
Cluster cluster = CouchbaseCluster.create("127.0.0.1");
Bucket bucket = cluster.openBucket(BUCKET_NAME);

// Here, the BUCKET_NAME = “prodcat” (The one that was created using the Couchbase admin console)

String key = "idGeneratorForProducts";
try {
    bucket.remove(key);
} catch (DocumentDoesNotExistException e) {
    
}

try {
    bucket.counter(key, 0, 20);
} catch (DocumentDoesNotExistException e) {
    log.info("counter doesn't exist yet and no initial value was provided");
}

At this point, our counter document is initialized to a value of 20.

We run the following code in a loop to insert product data in a sequential fashion:

long nextIdNumber = bucket.counter(key, 1).content();
log.info("nextIdNumber = "+ nextIdNumber);
String id = "Prod::" + nextIdNumber;
//you're now ready to save your document:
   Product product = ProductUtil.getProduct(nextIdNumber);
   JsonObject content = JsonObject.create()
          .put("type", Product.TYPE)
          .put("id", product.getId())
          .put("description", product.getDescription())
          .put("price", product.getPrice());
         
bucket.insert(JsonDocument.create(id, content));

Let's dive into the code above. The nextId is calculated by incrementing the counter by 1. We make use of the nextId to populate the “id” field of the product document. After completing the operation for three product documents, the prodcat bucket looks like this:

“idGeneratorForProducts” is the counter document that holds the current value of the counter. Each product document has the “id” populated with the sequence:

{
    "id": "Product 21",
    "price": "20",
    "type": "Product",
    "description": "This is a utility product"
}

It’s worth mentioning that we can implement the sequence in decreasing order as well. In that case, the only thing we need to do is:

• Initialize the counter to the maximum value.
• Decrement the counter by 1 to generate the nextId in sequence.
• Use the nextId to insert a document in a bucket. 

Conclusion

The code used for this article is written in Java and makes use of the Spring Boot and Spring Data Couchbase dependencies. The same concepts can be applied to any Couchbase client SDK. 

Source code for the application can be found at https://github.com/ratchakr/prodcat.