DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Data
  4. Modernizing Long-Running Transactions for the Microservices Era

Modernizing Long-Running Transactions for the Microservices Era

This open-source platform solves the long-running process problem in a modern microservices environment by replacing Sagas with workflows.

Dominik Tornow user avatar by
Dominik Tornow
·
Sep. 13, 22 · Opinion
Like (2)
Save
Tweet
Share
5.01K Views

Join the DZone community and get the full member experience.

Join For Free

While some software transactions are simple, others require more maintenance. Long-running transactions in particular have become the bane of the microservices era, as they need careful stewardship to ensure successful completion. Developers have dealt with this problem for decades by writing extra code to manage more demanding transactions. Now, in the distributed world of microservices, they must tackle it again. So exploring this topic — and how modern execution platforms can help — offers a number of compelling insights.

How Do We Define “Long-Running Transaction” Anyway?

The simplest transactions are predictable, involving a straightforward request and response interaction. However, more complex transactions span multiple steps that can take different lengths of time to complete, especially when manual input is involved.

An example of this is booking a multi-leg air flight. A person might need to book multiple flights as part of a single transaction, but it might take time to manually check and book options for each individual leg of the journey. So the system must keep track of what’s already been booked and what’s available, as the user goes through the booking process from start to finish. 

Careful design in a distributed application, comprising multiple microservices, is essential as the booking transaction might last mere minutes — or it could take days — depending on how long the user takes to complete their booking. Another example of a process, which often spans years, is a subscription management transaction. Streaming services like Netflix might charge subscribers each and every month. The system wakes up after 30 days, processes a credit card payment, and updates a customer’s account, in a series of regular smaller steps.

Both of these examples, airline booking and subscription services, show that it isn’t appropriate to define a long-running transaction based on the time it takes to execute. One might last a few minutes while the other spans years. So ultimately, uncertain execution times mean that long-running transactions cannot and should not have any time limits at all. For this reason, at Temporal, we call long-running operations arbitrary length operations. 

The Importance of Strong Execution Guarantees

One thing a long-running transaction must have is a strong execution guarantee, minimizing the risk of failure. This means that it must be stateful. If any underlying step fails, due to technical or manual disruption, it should recall which step it’s at, and resume operation from that point.

The airline booking app needs to either succeed completely, with all flights booked — or not succeed at all, with no flights booked in order to maintain the integrity of the user’s trip.

The subscription management transaction must also maintain state. It must remember that the user paid their subscription for June so that it can charge them in July. If it fails, it can’t simply restart from the beginning and recharge them for each and every month since the account was created. 

Short-running transactions, on the other hand, only provide weak execution guarantees. They can’t ensure reliability because they can’t maintain their own state in the event of a failure. Instead, they rely on the processes that called it to monitor its state, detecting and mitigating any problems with its operation.

Sagas and the History of Microservices

When we look back through history, a paper published at Princeton University way back in 1987 combined long-running and short-running transactions in a software development pattern known as a Saga. A Saga breaks down a long-running transaction’s steps into a series of short transactions.

The Saga manages each of these short transactions, issuing compensating transactions for those that fail. It maintains state across all of the short-running transactions using database storage for persistence. The Saga retries short transactions when necessary, based on message queues and timers.

Developers today use Sagas to support long-running transactions in microservices, but it’s painful. Sagas are patterns, not solutions, and developers must implement Sagas themselves. They must write code to address process failures, store state, and manage time limits. This is both burdensome and brittle.

Microservices make Sagas even harder to support because these services can call each other. A microservice conducting a short-running transaction might call another microservice for completion. The developer has to manage those relationships to support the overall Saga.

Workflows to the Rescue

Temporal is an MIT-licensed open-source platform, and solves the long-running process problem in a modern microservices environment by replacing Sagas with workflows. These are multi-step processes designed specifically for distributed systems that can last for indeterminate amounts of time.

Workflows exist independently of request-response flows while still providing strong execution guarantees. This means that the developer can support long-running transactions in their application without writing ad hoc state management code. The application needn’t be aware of failure. Temporal manages application state under the hood, regardless of how long that process runs.

Temporal’s environment includes redundant components and geographic independence across multiple availability zones to ensure consistency. Even in a catastrophic system state, the platform will preserve a workflow’s state until the underlying infrastructure recovers.

Not only does a workflow preserve its own state, but it also supports the chains of short transactions created when microservices call each other. If any of the microservices in a chain of short transactions fails, the platform can automatically restart them to help the chain complete.

Workflows achieve the same goals as Sagas, but with almost no work on the developer’s part, save for writing a few lines of code to set up the workflow. Just as a database hides time limits and failures from a developer when dealing with atomic transactions, a workflow hides them from developers in a microservices environment.

By managing application state as a core abstraction, Temporal modernizes the Saga solution for the microservices age. It lets developers create long-running transactions spanning multiple steps and hundreds of microservices with unpredictable execution times. This leaves programmers free to code their business logic safe in the knowledge that those workflows will stay consistent.

Whether your system is booking an air flight or managing a subscription that queries multiple systems — or any number of other examples — you can create a workflow in just a few lines of code. 

Coding and managing long-running transactions in complex applications needn’t be such a Saga after all.

microservice Data Types

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Remote Debugging Dangers and Pitfalls
  • Load Balancing Pattern
  • The 12 Biggest Android App Development Trends in 2023
  • Tech Layoffs [Comic]

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: