Full ACID and Transactions on Your Microservices

ACID implies Atomicity, Consistency, Isolation and Durability, and it's at the heart of being able to create transactions, where either the entire process is successful, or the entire process is discarded. To give you an example of why this is important, imagine you have an application which allows people to order things online. Typically its purchasing process would look like this.

1. Customer clicks the buy button
2. The order is internally handled and shipped
3. The system deducts money from the customer's bank account

If step 3 above fails, you have a problem. Maybe the customer didn't have sufficient finances to make the purchase? Still step number 2 was already executed at that point, and the order had been handled and processed. Your system has now entered an undetermined and invalid state, resulting in you probably loosing money.

For the above example, the fix is really simple. Just make sure you reorder your steps, such that the process becomes the following instead.

1. Customer clicks the buy button
2. The system deducts money from the customer's bank account
3. The order is internally handled and shipped

Typically deducting money from a bank account is an external service, often provided by a third party payment service providers. While the internal handling of the order, is an internal process, where state changes are applied to the internal system. And in fact, a general "design pattern" can be deducted from the above, which is as follows.

Always change state in your internal systems as the last step

If you follow the above simple rule, you've eliminated 50% of the problems related to partially failed micro service systems invocations, where you're dependent upon several sub-processes being able to finish their tasks, before the task as a whole is considered a success.

Idempotency

Idempotency is a really cool word. It implies that given the same input arguments, a function invocation will only change state the first invocation. The most famous use of this word, is in the HTTP standard and how it describes the PUT verb. The PUT verb should not apply further state changes in a system, if it's given the same payload twice. Only the first invocation should apply state changes.

In more complex micro service processes, the code executing when a client clicks the "Purchase button", typically invokes several micro services. Imagine something as follows.

1. User clicks the purchase button
2. Money is deducted from the client's account
3. Money is transferred to the manufacturer's account
4. The order is handled internally, and shipping is started

If step 3 or step 4 fails in the above process, you cannot simply rerun the process, because this would result in deducting the client's bank account twice. If all of the above invocations are idempotent though, the process can easily be re-animated by simply running it in its entirety once more. Idempotency is actually very easy to implement. Just create a new Guid or Uuid, and associate with every invocation to your micro service invocations, and if the micro service endpoint encounters an identifier it has previously executed, it returns early, and doesn't apply any state changes to its internal states. This of course requires persisting the identifier for the execution somehow, but that's fairly simple in practice. If the client calling the endpoint requires access to the returned value, you can also persist the return value of some micro service process, and return the previous result, looking it up using the invocation identification as a criteria.

This doesn't result in full transactions and ACID on your micro service architecture, and in fact that is not even possible in theory, due to the CAP theorem - But it eliminates 98% of the problem, especially resulting from transient errors, allowing the client in your micro service system to become oblivious to the internals of your system, assuming he wants to reanimate and retry a transaction object, if it fails due to a transient error. Transient errors here of course being typically 5xx errors in HTTP, where it's assumed that if the same payload is sent again, the process might probably succeed.

Basically, to sum things up, this relies upon two simple tricks.

1. Apply state changes to your internal systems last
2. Create your micro services as idempotent

And you've got "98% ACID" on your micro services. Now of course, what would a software development article be without example code ...

Guid.NewGuid();

And that's it :)