Why We Need a New Breed of Hybrid Microservices Platform

Many enterprises have been using both monolithic and microservices architectures, side by side, for their IT needs. Each architecture has its own benefits and challenges, yet neither one provides an all around solution to modern day IT challenges which demand high performance without compromising on the decoupling and agility offered by microservices. In this post, we’ll discuss the evolution of persistence technologies, the pros and cons of microservices versus monolithic architecture, and why a hybrid microservices platform is the way to go.

The Evolution of Persistence Technologies

The evolution of storage volumes and the increase in high-end transient processing engines, consisting of cache and RAM, has brought us to the era of In-Memory Computing (IMC). This golden age of IMC includes the evolution of Low Latency Distributed Microservices (LLDM).

These days, the boundaries between technologies aren’t very clear. Sometimes you find yourself using multiple technologies within a single project, such as RDBMS, NoSQL, caching technologies, IMDG, and more.

RDBMS is simple enough to understand, while NoSQL is still trying to figure out its place in the data world having Document (e.g. MongoDB), Key-Value (e.g. Coherence, Gemfire, Hazelcast, Gridgain, etc.), Columnar (e.g. Vertica, ActivePivot, etc.), and Graph (e.g. Neo4J) used in many different projects, sometimes in conjunction.

Finally, caching technologies, such as Memcached and IMDGs, have been around for the past decade. They’ve helped solve performance challenges that other storage volumes have not been capable of handling.

The downside to these technologies is that they introduce new challenges.

One of these challenges is eventual consistency, which is a consistency model used in distributed computing to achieve high availability. It guarantees, informally, that if no new updates are made to a given data item, eventually all accesses to that item will return the last updated value. The problem with this model is that many systems need an always consistent model because the data always needs to show the most recent value. If we take for example a bank account, we cannot build the system on top of a solution that will at some point be inconsistent.

The disadvantage of non-transactional platforms (Cassandra, for example) is that although one could avoid the locking of shared resources by linearly scaling out the data across the cluster (assuring availability), it comes at the cost of consistency.

Understanding this makes it clear why we need a new breed of distributed services platforms for a converged microservices architecture.

Microservices vs. Monolithic Architecture

The spectrum of monolithic and microservices is vast — the main challenge is using only the good qualities from each. At first glance, it would seem impossible to use elements from one end of the spectrum without giving up on others, but is that so?

Monolith Recap

To understand where we are heading, we need to take a look at common practices used in today’s business world. Looking at some of the more conservative verticals such as Financial Services, Telecom, Healthcare, and eCommerce, we see a pattern that they are reluctant to sway away from legacy Monolithic architectures due to their advantages. 

Monolith Pros and Cons

Pros:

• Classic monolithic architectures provide better performance and strong coupling.
• Less reliance on third-party or other department’s services.
• Full control of your application.

Cons:

• No agility for fast deployment or scalability.
• No simple way to make it highly available.
• Nearly impossible to isolate, compartmentalize or decouple system functionalities.

Microservices Recap

Microservices is many things, one of which is the ability to decouple services into smaller components having generic APIs and technologies where each service is isolated and contains the whole stack it needs to independently deploy and operate decentralized. To expand on the concept, take a look at Martin Fowler’s fantastic article on microservices.

Microservices Pros and Cons

Pros:

• Compartmentalization and decoupling: Allow you to scale up-and-out dynamically and on-demand.
• Agile methodology: Allows you to easily make changes to the code and run tests to ensure that everything is running smoothly. Each service is separated so you can change more than one at each iteration (unlike monolithic applications where everything is tied together, including the tests).

Cons:

• Granularity control.
• Inter-process and unsafe distributed communication mechanism.
• Handle partial failure — no transaction safety (eventual consistency).
• Update multiple databases owned by different services (polyglot persistence drawback).
• Multiple APIs.
• Refactoring can be quite difficult.
• Service discovery (phonebook service).
• Performance impact caused by HTTP, de/serialization, and network overhead.
• Achieving and maintaining high availability is no easy task.

What’s Next: A Hybrid Approach Is the Way to Go

A full monolithic approach isn’t sufficient anymore for systems that need scalability, such as IoT. Also, a full microservices approach isn’t great for performance or managing the services’ life-cycle.

We see the next logical step as a hybrid approach — utilizing monolithic performance with all the advantages of microservices, all while looking forward to the future of HTAP for the true harnessing of data and event-driven architecture. And that’s exactly where we’re heading.

The Perfect Platform to Achieve High-Performance Distributed Microservices

The XAP platform is not a monolith; it’s an interoperable means to an end of how to manage the services zoo we’re seeing growing out of control in every major enterprise. Service-oriented architecture, event-driven architecture, and, most significantly, data-driven design, are hot topics these days. Given that there are common architectures, topologies, and solution platforms in the market, we need to ask ourselves, how do we build the most efficient, cost effective, and high-performance system possible to handle the ever growing real-time challenges?

XAP Microservices Platform is the only way to combine the benefits of the monolithic approach with all the advantages of microservices. XAP is a low latency, distributed microservices platform comprised of a cluster of machines that work together to create a resilient shared data fabric for low-latency data access and extreme transaction processing. Despite the fact that it runs on a cluster of machines, achieving distributed service-like processing tasks and data partitioning, the underlining infrastructure is only one platform. It may seem like a Monolithic platform from the outside, but it most certainly is not. 

6 Reason to Use XAP’s Microservices Solution 

1. Granularity control: Building microservices by using small granular components is a hassle, a management nightmare, and a performance no-no. XAP's microservices distributed platform is a no-compromise approach where one can decouple the services and increase performance at the same time.

2. Eliminates the need for communication between different processes: By addressing the first issue, granularity control, we’ve actually solved many issues out-of-the-box. Communication between different processes, though still possible with cross partition querying, is no longer needed once the logic, messaging, and the (relevant) data are coupled together. 

3. Database/remote site full consistency: Eventual consistency is a big design flaw in most systems that requires data accuracy and consistency. Consistency is maintained through all levels of the platform. Transactions may span multiple partitions, multiple data types (Customer, Product, etc.), and multiple platforms across different sites if necessary. A transaction may start on the client-side, continue on the server-side with some collocated logic, and end back at the client-side if required. Since XAP acts as the system of record (application data fabric), it must support ACID transactions including database persistency and remote site replication atomicity.

All distributed transaction items that may spread across multiple nodes are transported to the database or remote sites as one unit ensuring database/remote site full consistency. 

4. Asynchronously update multiple storage volumes: Having a platform that can support services and data for extreme microservices performance is necessary for key systems that are built for performance—i.e. payment processing, trading, IoT, customer 360, passenger re-accommodation, healthcare management, etc. Although a strong consistency is a must, often we see a need to replicate the data to storage volumes either for persistency or because a third-party application can only work with a specific storage volume. XAP can asynchronously update a variety of common storage volumes out-of-the-box, making such a task a breeze for any developer that wants this feature to be integrated with their microservices architecture. 

5. Performance impact: A microservices platform needs to support the following hybrid cloud architectures as a service: IMDG, Analytics, Compute Grid, Caching, Replication. Only the XAP microservice architecture can tackle all of the above in one platform by using a unique mechanism called a deployment unit (DU) or processing unit (PU) underneath the hood. The XAP PU is the unit of scalability and failover. The PU supports both Java and .NET. It includes the definition of artifacts that the user would like to have in a common lifecycle. These may have some dependency. For Java-based PU, such dependency would be described via Spring IoC. XAP allows you to deploy multiple independent PUs into the same grid or deploy a composition of PUs that have some inter-dependency. In such a case, XAP will orchestrate the PUs provisioning to have the right deploy, healing, and scaling order. 

6. Orchestration, visualization, and beyond: Whichever orchestration tool you choose, XAP is the exact layer to be used underneath as the microservices platform. It can collocate the data, logic, and messaging together for extreme performance yet decoupling the distributed encapsulated services within the grid. As complicated as that sounds, the grid does all the work solving most if not all of the microservices cons. Using any cloud or VM/Openshift/Kubernetes/Docker solves one half of the puzzle, while attaching the XAP microservices platform is the other half. 

Implementing microservices oriented systems across industry verticals requires special attention to performance and scalability. It’s nearly impossible to implement a real-time microservices architecture if you rely on caching, database, and messaging systems to act as your data state management and transfer fabrics. 

With XAP, the platform powers the delivery of real-time microservices that may run within your existing Iaas/PaaS as a first class citizen delivering cost-effective, agile, event-driven applications that never fail. 

Final Thoughts

We’ve seen that neither a fully monolithic or microservices approach is enough for complex enterprise systems. However, most enterprises are still unwilling to decommission their core monolithic systems due to the high price of performance loss. Yet the shift that XAP provides to a fully usable enterprise microservices data fabric while being distributed, secure, and highly available — can make all the difference. Together with the cloud-native approach, our hybrid solution delivers strong consistency, highly available, distributed, scalable, encapsulated services deployment infrastructure.