Microservices is a buzz word at the moment. Is it really something original or is it based on established best practices? There are some disadvantages to the way microservices have been implemented, but can these be solved?
Component Testability and Consistency
Once you have assembled a large system, it can be hard to impossible to profile where the highest delays come from. You can profile for average latency or throughput, but to achieve consistent latencies, you need to analyze key portions of your system. This is where having simple components which run independently can help the consistency of your system end to end.
The UNIX Philosophy
Many of the key concepts of microservices have been used in distributed systems for many years. Microservices have much in common with the Unix Philosophy.
Mike Gancarz is has summed up these principles:
Small is beautiful.
Make each program do one thing well.
Build a prototype as soon as possible.
Choose portability over efficiency.
Store data in flat text files.
Use software leverage to your advantage.
Use shell scripts to increase leverage and portability.
Avoid captive user interfaces.
Make every program a filter.
The Microservices Architecture is the UNIX Philosophy applied to distributed systems.
The philosophy of microservices architecture essentially equals the Unix philosophy of "do one thing and do it well." It is described as follows:
- The services are small—fine-grained to perform a single function.
- The organization's culture should embrace automation of deployment and testing. This eases the burden on management and operations.
- The culture and design principles should embrace failure and faults, similar to anti-fragile systems.
- Each service is elastic, resilient, composable, minimal, and complete.
There are disadvantages to using a microservices arhictecture some of which are:
- Services form information barriers.
- The architecture introduces additional complexity and new problems to deal with, such as network latency, message formats, load balancing, and fault tolerance; ignoring one of these belongs to the "fallacies of distributed computing."
- Testing and deployment are more complicated.
- The complexity of a monolithic application is only shifted into the network, but it persists.
- Too-fine-grained microservices have been criticized as an anti-pattern.
Can we get the best features of a monolith and microservices? Does it have to be one or the other? Should we not use the approach which best suits our problem? One of the key aspects of microservices is controlled deployment of an application. So shouldn’t we be able to deploy components as a monolith or microservices where it makes the most sense to do so?
Proposed alternatives to microservices include:
- Package the functionality as a library, rather than a service.
- Combine the functionality with other functionality, producing a more substantial, useful service.
- Refactor the system, putting the functionality in other services or redesigning the system.
How We Can Get the Best of Both Worlds
Make Your Components Composable
If your components are composable, they are always the right size. You can combine them as needed into a collection of services or everything into one service.
This is particularly important for testing and debugging. You need to know a group of components work together without the infrastructure getting in the way. For the purposes of a unit test, you may want to run all your components in one thread and have one directly call the other. This is not more complicated than testing components of a monolith where you can step through your code from one component to another and see exactly what is happening.
Only once your components work together correctly without infrastructure do you need to test how they behave with infrastructure.
Make Your Infrastructure as Fast as Your Application Needs
Low latency trading systems are distributed systems, and yet they also have very stringent latency requirements. Most trading systems are designed to care about latencies much faster than you can see. In the Java space, it is not unusual to see a trading system which needs to have latencies below 100 micro-second 99 percent of the time or even 99.9 percent of the time. This can be achieved using commodity hardware in a high level language like Java.
The keys to achieving low latencies are:
- Have a low latency infrastructure for messaging and logging—ideally around one micro-second for short messages.
- A minimum of network hops.
- A high level of reproduce-ability of your real production load so you can study the 99 percent (worst one percent) or 99.9 percent (worst 0.1 percent) latencies.
- Viewing each CPU core as having a specific task/service, with its own CPU cache data and code. The focus is on the distribution of your application between cores (rather than between computers).
Your L2 cache coherence bus is your messaging layer between high performance services.
You can perform a CAS operation on the same data between two difference cores, where each thread toggles the value set by the other thread with a round trip time of less than 50 nano-seconds on Sandy Bridge processors, and even less on newer generations.
Examples of low latency instructure in Java are:
These transports have different advantages in terms of handling load balancing and failover.
Make the Message Format a Configuration Consideration
There are a number of competing concerns in message formats. You want:
- Human readability so you can validate that the messages are not just behaving correctly by doing so in the manner you expect. I am often surprised how many issues I find by dumping a storage file or the message logs.
- Machine friendly binary format for speed.
- Flexibility in terms of future schema changes. Flexibility means adding redundancy so the software can cope with adding/removing fields and changing their data types in the future. This redundancy is a waste if you don’t need it.
Ideally, you can choose the best option at testing/deployment time.
Some examples of serialization libraries where you can change the actual wire format to suit your needs are:
- Jackson Speaming API, which support JSON, XML, CSV, CBOR (a binary format)
- Chronicle Wire, which supports object serialization YAML, a number of different forms of Binary YAML, JSON, CSV, raw data
What I found useful in YAML versus JSON is the cleaner syntax, which is designed to be human readable, rather than the subset of another language, the natural support for data types, comments, binary content, and message seperators.
I think there are a lot of good ideas on how to use microservices, and I think many of the criticisms around them are based on how they have been implemented. I believe those problems are solvable.