Saga Pattern: Compensating Distributed Transactions

In today's web world with an increase in the load to monolithic web application a new system design came into existence that is called microservice architecture design. Microservice architecture design helps us to structure an application as a collection of services that are loosely coupled and independently deployable. As there are many benefits of using microservice architecture but it too comes with some challenges. One of the main challenges is to handle the distributed transactions.

In this tutorial, we will discuss about the distributed transactions and how to handle it using the Saga pattern.

Assuming we have three microservices (service, service2, and service3) and based on some transactions that happened in service1, it will trigger an event which calls the next microservice i.e. step 2 and similarly service2 will call service3 microservice.

1. service1

2. service2

3. service3

and we have a distributed transaction use case.

1. If service1 transaction succeeds call service2 and if service2 transaction succeeds call service3 and if service3 transaction succeeds mark the transaction as completed.

2.If service1 transaction succeeds call service2 and if service2 transaction fails mark the transaction as failed and rollback the service1 transaction.

3.If service1 transaction succeeds call service2 and if service2 transaction succeeds call service3 and if service3 transaction fails mark the transaction as fail and rollback the service2 and service1 transaction.

4.If service1 transaction fails mark the transaction as fail and rollback the transaction.

This is where Saga pattern comes to rescue.

Saga pattern is an asynchronous and event-driven design pattern which consists of a sequential/parallel chain of microservices (steps) which are been called based on some events(transactions) that occur from within a single microservice.

We can also say Saga pattern is a fail-over and compensating handling pattern, so whenever there is any failure, it will execute the corresponding compensating action to return the initial state from where the Saga flow started.

So to implement the saga pattern one of the best solution is to use AWS step function.

Lets understand what is Step function and  how it can fit to our use case.

Step functions are basically an orchestration service for AWS Lambda and activity-based tasks.It makes easy to coordinate  microservices using visual workflow.

• State machine: In simple word the workflow which we build using step function is called a state machine. It describes how to propagate inputs from one state to the next
• AWS States: It's a way to tell the state machine to execute some task and there are seven types of AWS states you can have:

1. Task

2. Pass

3. Wait

4. Succeed

5. Fail

6. Choice

7. Input and Output

For more details about the task please refer the AWS - AWS Step Function

Now as we understand state machine consist of different states and states consist of task.

Let's develop the state machine for our use case.

Firstly we will create Service1 as one of the state as mention  in below code

"States": {

    "Service1": {

      "Type": "Task",

       "Resource": "<Transcation event for service1>",

      "Catch": [        

        {

          "ErrorEquals": ["States.ALL"],

          "ResultPath": "$.Service1Error",

          "Next": "rollback Service1"

        }

      ],

      "ResultPath": "$.Service1Result",

      "Next":"Service2"

    }

The notable part is Type, Resource (It is used to trigger Transaction event for service1 mostly done via calling AWS lambda function), Catch(this is used to rollback the transaction occurred by service1, for achieving the same we create the new state i.e. rollback Service1), ResultPath(It is helpful for passing the output of first state to another state), and Next(It is use to call the next state from existing state)

Similarly, we can develop the states for Service2 and Service3 along with the rollback states for both the services.

Below I am sharing the final state machine JSON along with AWS generated visual workflow with all the scenario which we have discussed at the starting of the discussion.

I hope now you are able to understand why we called it a compensating transaction.

xxxxxxxxxx

{

  "Comment": "Saga pattern Using Step Functions",

  "StartAt": "Service1",

  "States": {

    "Service1": {

      "Type": "Task",

       "Resource": "<Transcation event for service1>",

      "Catch": [        

        {

          "ErrorEquals": ["States.ALL"],

          "ResultPath": "$.Service1Error",

          "Next": "rollback Service1"

        }

      ],

      "ResultPath": "$.Service1Result",

      "Next":"Service2"

    },

    "Service2": {

      "Type": "Task",

       "Resource": "<Transcation event for service2",

      "Catch": [        

        {

          "ErrorEquals": ["States.ALL"],

          "ResultPath": "$.Service2Error",

          "Next": "rollback Servicer2"

        }

      ],

      "ResultPath": "$.Service2Result",

      "Next":"Servicer3"

    },

    "Servicer3": {

      "Type": "Task",

      "Resource": "<Transcation event for service3>",

      "Catch": [        

        {

          "ErrorEquals": ["States.ALL"],

          "ResultPath": "$.Servicer3Error",

          "Next": "rollback Servicer3"

        }

      ],

      "ResultPath": "$.Servicer3Result",

      "End": true

    },

    "rollback Service1": {

      "Type": "Task",

      "Resource": "<Transcation event for rollback service1>",

      "ResultPath": "$.rollback Service1Result",

      "Next":"Fail"

    },

    "rollback Servicer2": {

      "Type": "Task",

       "Resource": "<Transcation event for rollback service2>",

      "ResultPath": "$.rollback Servicer2Result",

      "Next":"rollback Service1"

    },

    "rollback Servicer3": {

      "Type": "Task",

       "Resource": "<Transcation event for rollback service3>",

      "ResultPath": "$.rollback Servicer3Result",

      "Next":"rollback Servicer2"

    },

    "Fail": {

      "Type": "Fail"

    }

  }

}