From Modules to Microservices via a Service Mesh

The title says it all, but details are always welcome. But first some definitions. According to the classic article by Martin Fowler:

“The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable.”

A monolith on the other hand is an application deployed as a single logical executable. The problem with a monolith is that, again quoting Fowler,

“Change cycles are tied together - a change made to a small part of the application, requires the entire monolith to be rebuilt and deployed. Over time it's often hard to keep a good modular structure, making it harder to keep changes that ought to only affect one module within that module. Scaling requires scaling of the entire application rather than parts of it that require greater resource.”

Continuing,

“For as long as we've been involved in the software industry, there's been a desire to build systems by plugging together components... A component is a unit of software that is independently replaceable and upgradeable…

We define libraries as components that are linked into a program and called using in-memory function calls, while services are out-of-process components who communicate with a mechanism such as a web service request, or remote procedure call.”

Thus, a microservice architecture (MSA) is a decomposition of an application into components as services. The advantages of an MSA are the components have firm module boundaries which can be developed by independent teams. They are independently deployable and scalable (thus increasing the feature velocity) and can be built in different languages, such as Nodejs and Java.

I have a sneaking suspicion that it is the componentization of applications that is the principal draw of an MSA. The main problem in developing an application using an MSA, either as a greenfield application or migrating an existing application, is in determining the component boundaries. For all the hand waving with Domain-Driven Design/ Bounded Context, this can be a non-trivial task requiring several rounds of refactoring particularly to decouple the persistence layer [Sam Newman]. This is made all the harder when the components are spread over network boundaries. I would suggest that is because we are missing the intermediate step of modularization of our application.

A look at the dates tells the story: The aforementioned Fowler article, introducing MSA, was in 2014. Sam Newman’s classic book was published in February 2015. Java 9 with Java Platform Module system was only released in 2017. So, at least for Java developers lacking a module system, MSA provided a substitute.

The main advantages of a module system are:

• Strong encapsulation behind explicit interfaces: The implementation is hidden behind well-defined interfaces. This what allows different teams to develop different modules independently.
• Reliable configurations: This allows us to be explicit about the module dependencies. Hence, it allows us to determine missing dependencies at compile time. This is a significant weakness of MSA where the dependencies are not explicit and missing dependencies are detected only at run time.

Modular Applications

A modular application is a collection of modules communicating via well-defined interfaces possibly developed by independent teams. Starting in Java 9, modularity is in the platform itself. Other languages have equivalents, for instance, JavaScript has had modules since ES2015. Modules written in different languages can run on a polyglot VM such as GraalVM. GraalVM’s polyglot capabilities make it possible to mix multiple programming languages in a single application while eliminating any foreign language call costs. Some of the languages supported by GraalVM are the JVM languages, JavaScript, Python, Ruby, and LLVM languages such as C and C++, and many more.

A modular monolith is a modular application deployed as a single executable running possibly on a polyglot VM.

If the application is a modular monolith, it is comparatively easy to refactor the component boundaries, particularly in Java since the module system has compiler support. If after repeated architectural refactoring we firm up the boundaries and determine which collection of modules — call it a microlet — needs to scale up, it is time now to peel off that microlet and make it a separate deployment unit. Now the communication is no longer an in-process communication but rather a remote communication via HTTP or gRPC. Of course, you need to modify your code to convert the local calls to REST.  But that is not all. Replacing local calls with remote calls runs into the well-known fallacies of distributed computing.

• Network is reliable — Provide for automatic retries.
• Latency is zero — Provide for request timeouts, circuit breakers.
• Network is secure — Provide for certificates and mutual TLS.
• Network is homogenous — We need to use tools like Jager to trace the call.

If like me, you are used to developing your microservices using the Spring Boot and Netflix stack then all this is in the code which we have to modify. But we can externalize the network concerns.

Service Mesh

A service mesh is a connective tissue between your services that adds additional capabilities which allow applications to offload these capabilities from application-level libraries. In a service mesh like Istio, a proxy is manually or automatically injected into the pod that houses your microservice. This 'side car' intercepts all the inbound and outbound calls to apply policies like retries, timeouts, circuit breaker, client-side load balancing. All HTTP calls can be configured to use mutual TLS. In addition, it supports Jager for tracing and integrates with tools like Grafana and Prometheus for metrics. All this is done via configuration in yaml files using the istioctl tool. See this for details.

Conclusion

The essence of the matter is that modularization is a design/development time activity and distribution over the network a deployment time activity. Missing the modularization step, most discussions of MSA conflate the two. This separation leads to a more straightforward road map. In a nutshell: First Modularize, then (if needed) Distribute.