Reactive Persistence for Reactive Systems

This article is featured in the new DZone Guide to Microservices. Get your free copy for more insightful articles, industry statistics, and more! 

A reactive microservices architecture is an architectural style that strives to provide the highest levels of responsiveness, resiliency, and elasticity, and accomplish this by adopting strong decoupling, isolation, non-blocking, event-driven architecture, and asynchronous messaging, among other techniques.

Traditional Persistence Is a Potential Bottleneck for Reactive Systems

Reactive architecture uses Domain Driven Design (DDD) or similar design patterns to accomplish isolation and create separate and isolated entities with aggregate routes and bounded contexts to create well-rounded and independent microservices.

By persisting such separated entities as part of business transactions using traditional persistence methods within the conventional CRUD realm, such as Two Phase Commit or 2PC, and adhering to the ACID concept, we are creating significant blocking bottlenecks that can potentially limit the capabilities and promise of reactive systems. Traditional persistence is not a good fit for reactive systems. We need a persistence approach that can promote minimal blocking and create decoupling between the business services and the persistence layer.

Reactive Persistence CQRS and Event Sourcing

Reactive persistence uses Command Query Responsibility Segregation (CQRS) and event sourcing to accomplish this asynchronous and decoupled interaction.

The CQRS will provide the decoupling between the read and write channels, allow greater freedom, and provide responsiveness for the persistence operations.

Event sourcing focuses on appending the various states of an entity or domain to an event journal using event persistence commands while you can have event subscribers and processors attached to the journal responding to the appended events and working on building the entity or object state in the query database or persistence store.

Event sourcing and CQRS will provide better write performance as it will only perform append operations instead of a full domain object update, and will improve scalability as we have decoupled and separated the Write and Read processes. Also, event sourcing provides better audit control and a ripe environment for analytics.

To drive the point closer, we can follow an order management example or use case. Using event sourcing, we can append different order states to the journal while we append similar events to other domains associated with an order, such as payments or inventory. The event subscribers and processors will construct the domain state in the query store using those fragmented events received from the event producers.

Technologies

DDD, CQRS, and event sourcing are not new design patterns. However, recent advances in network and persistence technologies allowed the open source community to build and implement those design patterns, providing robust and enterprise-grade, highly available, scalable, and distributed reactive persistence technologies.Below are two out-of-the-box technologies that fully or partially support CQRS and event sourcing.

Lagom Persistence

Lagom is a reactive framework built on top of the Akka Toolkit and Play framework technologies. The framework is created and supported by Lightbend.

Lagom Persistence is a CQRS and event sourcing implementation, and can easily provide direct mapping to the business domains by deriving and attaching the core Lagom persistence classes and utilities.

Apache Kafka

Kafka is another revolutionary technology, it’s a high-performance and high scalability distributed data streaming platform.

Kafka is a clustered and distributed pub/sub asynchronous messaging platform and event store at the same time. As highlighted in Figure 1, we can use Kafka as the event store and separate the front-end services supplying domain events through commands (producers) from the backend services that subscribe to those events and construct the domain state in the query database store.

Reactive Persistence Consideration

When using CQRS and event sourcing, we have some important aspects to consider, some of the most important ones are described further below.

Eventual Consistency

Reactive persistence is using eventual consistency rather than strong consistency. For instance, using the order management example we used earlier, the order status (created, submitted, completed, etc.) might not be consistent with the event store’s latest event for some time, and the business logic needs to compensate for and accommodate eventual consistency.

Entity Tracking

Using the order management example, all the events are stateless and immutable; so how can we keep track of the order? The system or the services must keep the order’s unique id in the exchanged messages, and the events must be able to identify and construct or reconstruct the specific order.

Order of Events

In a distributed system, the order of events is of significant importance, as constructing the correct domain state will directly depend on the correct order of events. For instance, if the event processor processes the “order submitted” event after the “order completed” event, the wrong order state will be represented in the order table.

Conclusion

Currently, we have a variety of technologies and techniques used in designing and implementing reactive persistence. Those different approaches can fit different use cases; each has their own advantages and trade-offs.It’s up to the architects to determine the best fit. However, the good news is that we are not bound to traditional persistence anymore, and we have many options where we can easily build and utilize reactive persistence.

This article is featured in the new DZone Guide to Microservices. Get your free copy for more insightful articles, industry statistics, and more!