Distributed Multi-Document ACID Transactions in Couchbase

The Couchbase CTO, Ravi Mayuram, announced the Beta of Distributed Multi-document ACID Transactions in Couchbase Server 6.5. I highly recommend reading Ravi's blog, which highlights how Couchbase transactions are an innovative union of ACID guarantees with scale, high-availability, performance, and flexibility.

In this article, I will dive deeper into our distributed ACID transactions functionality.

Simple Yet Powerful

First off, let's look at how easy it is to write a Couchbase transaction. You open a transaction, do some work, and commit (or fail) with Atomicity, Consistency, Isolation, and Durability. Leveraging the regular Couchbase SDK and APIs, you can utilize the capabilities of the underlying programming platform. Here is a code snippet that shows a basic debit/credit by transferring funds from one account to another with an ACID transaction:

There are many capabilities built in to make programming with transactions in Couchbase frictionless:

• Automatic commit or rollback with the flexibility to also do this explicitly.
• Automatic retries for transient errors such as durability delays due to network glitches or lock conflicts.
• Configurable duration for a transaction. Deadlocks are automatically resolved since one of the transactions will timeout and release its resources.
• Automatic cleanup of orphaned transactions if the client crashes.

ACID Guarantees

Couchbase transactions provide Atomicity, Consistency, Isolation, and Durability, as the ACID guarantees require:

• Atomicity and Durability are binary properties — either you have it or you don't.
• Isolation is a spectrum with tradeoffs. Popular relational databases such as Oracle, MySQL, and SQL Server have chosen different default and maximum isolation levels.
• Consistency has different meanings to the traditional ACID audience and to the distributed systems audience. The traditional ACID definition says that a transaction should take the database from a valid state to another. The distributed systems definition stemming from the CAP theorem says each read should receive the latest write. Although, this starts to sound a little like Isolation semantics.

Here is a summary of the ACID guarantees provided by transactions in Couchbase Server 6.5.

Yes, They Are Distributed

Couchbase ACID transactions are multi-document and multi-node, so they are truly distributed, as you can see in the simplified Couchbase architectural illustration below. The illustration shows a simple Couchbase cluster with 3 nodes and 9 vbuckets/shards with 2 replicas (total 3 copies of data). The transaction modifies two documents, Andy and Beth, with their active copy on two separate nodes.

The successful completion of the transaction will ensure that the active copy of each of the two documents, as well as a majority of their total copies (which in this case means 1 of 2 replicas in addition to the active), are updated with the new value. Failure will guarantee that the state is unchanged and identical as prior to the transaction.

Feature Drill Down

While the ACID summary provides a high-level perspective, the following details provide a deeper understanding of the semantics:

Multi-Node, Multi-Bucket, Multi-Document

A single transaction can span multiple documents in multiple buckets on multiple nodes. Since, in Couchbase, a bucket maps to a database, a transaction can actually span multiple databases living on the same Couchbase cluster. This, along with Couchbase's ability to perform JOINs across buckets, gives enormous flexibility to the application.

Only Committed Data Is Readable by DCP Consumers

All reads done in any manner including from N1QL, XDCR, Analytics, Mobile, Eventing, and Connectors will only return committed data. No one will ever be able to read any uncommitted/dirty data. Couchbase Data Change Protocol (DCP) protocol ensures that no uncommitted, intermediate data is ever read by a downstream consumer.

Locking and Conflict Detection

Conflicts between transactions trying to update the same documents are detected and handled by rolling back one of them and retrying it (until the transaction timeout which by default is 15s). This is done via a combination of optimistic and pessimistic locking at the document level.

Any document you want to modify in a transaction has to be first read inside the transaction, and all writes to it are automatically CAS (Check and Set) writes. Hence, if a transaction reads data that subsequently gets changed by another transactional (or non-transactional) write, then at write time, this transaction will detect the CAS conflict, rollback, and retry.

In the debit/credit transaction above, where money is being transferred from Beth's account to Andy's account, if Beth's account changes after the transaction has read her account but before it has modified it, the transaction will detect that, rollback, and retry.

Once the document is modified in the transaction, it implicitly gets a write lock on the document, which is released only when the transaction ends. If a second transaction tries to modify a document that is already locked by a transaction, the latter transaction will detect the write conflict, do a rollback, and retry until the transaction timeout period.

Going back to the debit/credit example above, assume an in-flight transaction where Beth's account has been deducted but Andy's has yet to be updated. At this point, Beth's account document is locked. So, if another transaction to transfer money from Beth's account to Bill's account is being tried concurrently, it will detect the write conflict on Beth and rollback.

Synchronous Durability

In Couchbase Server 6.5, we have a new implementation of the replication protocol that ensures a mutation is visible to readers only after it has met its replication/persistence criteria and can be rolled back if those criteria are not met. This new replication mechanism is undergoing comprehensive in-house Jepsen testing.

Synchronous replication provides tunable durability, allowing you to choose the level of protection against failures (node crash, disk failure, single or multiple failures) that you want for each write. There are 3 levels to choose from:

• majority — this durability level ensures that the write is propagated to a majority of the replicas before it returns (e.g. if the cluster is set up with 2 replicas, it will be written to RAM on the active and propagated to at least 1 replica before returning success to the application).
• persistToActive — this level ensures that the write is propagated in RAM to a majority of the replicas and persisted to disk on the active before it returns success to the application.
• persistToMajority — this level ensures that the write is persisted to disk on a majority of the nodes before returning success to the application.

If durability fails (timeout, node failure, etc.), the write will be automatically rolled back and the client notified of the failure.

Note: The new synchronous durability writes are applicable to transactions as well as single document mutations. Note: If you do not specify any durability level then it is handled asynchronously which is eventually durable.

Transactions are layered over the new Synchronous Replication mechanism and, by default, set the durability level at 'majority'. You can override the default by choosing a different durability level per transaction. 'persistToMajority' provides the strongest data protection in case of multiple failures.

Ephemeral Buckets

You can use transactions as well as synchronous durability on Ephemeral buckets as well. There are caching use cases that do not need persistence but still want to ensure items in the cache are updated with ACID guarantees.

To learn more about Couchbase ACID transactions and other new features in Couchbase Server 6.5, the following blog is a good place to start: https://blog.couchbase.com/announcing-couchbase-server-6-5-0-beta-whats-new-and-improved/