Microservice Madness: Debunking Myths and Exposing Pitfalls

Microservice is the false belief that adding a message broker to your app will somehow magically make it faster and more scalable.

Ignoring the fact that this is, in itself, an oxymoron—and that your app quite literally becomes two billion times slower—the absolute dumbest argument I've ever heard in favor of microservices is:

"Microservices will decouple dependencies."

This argument is the equivalent of arguing in favor of decapitating heads to cure headaches. Sure, it will decouple dependencies, but only by literally spending 2,000,000,000 times more resources and 2,000,000,000 times more time for every single function invocation towards another "service."

This happens because as you "scaled out" that one function invocation by going through a message broker to another service having the responsibility to handle your function invocation, you rewired your application such that instead of spending 3 to 5 CPU cycles to invoke your function, it now requires serialization into JSON, transferring the payload over a socket connection to your message broker, deserializing the payload again, validating it, serializing it (again! sigh!), and transmitting it to your microservice server, which now again has to deserialize the JSON into an object that it can use to actually deal with your invocation.

Congratulations — you've taken a function invocation requiring 2 to 5 CPU cycles and turned it into a serialization-based socket monster, easily consuming 1,000,000 times more memory and around 2,000,000,000 additional CPU cycles!

Dependencies

Dependencies is the foundation every spaghetti app that ever existed rests upon. The less dependencies, the more easily maintained and "agile" your app becomes. Microservices will reduce dependencies, because it forces you to serialize your types into generic graph objects (read; JSON or XML or something similar). This implies that you can just transform your classes into a generic graph object at its interface edges, and accomplish the exact same thing.

Generic Graph Objects (GGO) == ZERO DEPENDENCIES!

Magic and Hyperlambda bring this idea to its extremes, because of its reliance upon Active Events. To eliminate dependencies however, you don't need sockets, message brokers, or even serializing your payload. All you need is the ability to generically transfer "data" from one component to another, where each component is a loosely tied together building block, running in-process.

Magic achieves this with its Node class. And there's nothing in this setup that requires sockets or serialization processes or deserialization processes. It's just a "mindset" in fact, so easily taught, that it can be summed up with a simple interface with a simple method, and a simple class with three fields.

Below you can find an implementation of the "last method, class, and interface you will ever need."

[Slot(Name = "foo.bar")]
public class FooBar : ISlot
{
    public void Signal(ISignaler signaler, Node input)
    {
        input.Value = 42;
    }
}

Magic's Node class again, is just a generic graph object (tree structure) that allows for passing data of any type around internally within its methods. Below is some pseudo code to illustrate the point.

class Node
{
    string Name;
    object Value;
    List<Node> Children;
}

The point about the above method is that the Node class indirectly contains all "arguments" to the method, both input arguments and returned arguments. Since objects are passed by reference in C#, this implies that an invocation only consumes four additional bytes to hold the reference pointing to your original object, while your serialization library for transferring JSON to another service, probably easily will consume hundreds of kilobytes, and sometimes megabytes to achieve the same. This is even before we start measuring the networking overhead!

This allows for implementations of the above method being able to deal with any type of objects you can imagine. Since all data at the fundamental level is simply a graph object, this allows two components to agree upon what input arguments to submit, and what output arguments to return, without neither of the components requiring any shared types besides the ISignal interface and its Node class.

This simple "trick" allows for completely decoupling every dependency between the "client code" and the "server code", without and networking or serialisation occurring

At this point, the smartest amongst us will claim that this is an unnecessary overhead to pay for being able to decouple dependencies, which they will be right about, but it is literally 2.000.000.000 million times more efficient than transmitting your payload to a microservice. And you can choose at which places in your app you want to use it! And if you, at some point in time, need to scale horizontally, you change the method invocation of your in-process function to invoke another server with its payload, returning the result of the invocation to caller.

Internally within the component itself, you wouldn't want to use this overhead — but then each class, method, and type you declare in that component, should also be explicitly internal — completely hiding the entirety of the component's internals, while exclusively using the Node class every time you need to interact with some other component.

This becomes the "superhuman equivalent" of encapsulation and cohesion, making even extremely strongly typed languages such as C++ and C# "blush"for their lack of encapsulation and cohesion ...

And because of its declarative nature, you've now reduced the size of your codebase (read; Technical Debt!) by 75% ...

Magic Cloud

Magic is entirely built upon this axiom, and in Magic it is taken to its extreme. If you clone Magic, you will rapidly realize that it's not one project, it's in fact more than 40 projects! And there are zero dependencies between these projects. Creating a new functionality for Magic doesn't even require changing any parts of its existing codebase. It only requires you to compile a DLL that's using Active Events, and dropping it into the bin folder of your application. With Magic, you don't even need your host process to reference your added functionality, because such assemblies will be dynamically loaded during startup of your application process ...

This is possible because all projects are based upon Magic Signals and Magic Node, and each "function" have the exact same (strongly type) signature, allowing you to literally exchange any function (in theory) with any other function.

Hyperlambda basically implements "implicit polymorphism" on every single method that exists in your app!

And within Magic as a project, you will find such "functions" with names such as while, if, else-if, etc - Basically all traditional programming constructs required to have a Turing complete "execution platform" allowing you to describe wanted functionality using intentional programming and declarative programming.

Hyperlambda

Since Hyperlambda again is literally nothing but the ability to serialize this graph object (the Node class), this allows you to express intent using a humanly readable text file format. Below is a small snippet of Hyperlambda to illustrate the point.

.data
   item1:Hello from Item1
   item2:Hello from Item2
   item3:Hello from Item3
for-each:x:@.data/*
   log.info:x:@.dp/#

This is because graph objects are implicitly serializable, similarly to how HTML, XML, YAML, and JSON is. But instead of using angle brackets and curly braces, it's using three SP characters. However, fundamentally, Hyperlambda is just a graph object file format, similar to XML, JSON, and HTML ...

Cold Fusion

This combination is basically the equivalent of "cold fusion for software development" because it allows you to infinitely scale complexity without risking that dependencies destroy your ability to scale — since, at any one particular point in time, you only have to think about a single project with some 100 to 500 lines of code — while "the app" as a whole, in theory, might contain hundreds of trillions of lines of code that you don’t have to worry about breaking as you're editing code, or causing harm to during execution or development.

In fact, this way of thinking about software development also — at least in theory — allows you to scale horizontally too, by creating software projects with millions of developers participating, without any of them needing to communicate with each other in any way, or collaborate or orchestrate their efforts in any other way. Because nobody is "building an app" — everybody is instead building individual, small pieces of "functionality" that are later orchestrated together to "become an app."

Basically, all the "promises" from microservices architecture, only 2 billion times faster, using 0.000000001% of the resources ...

And everything is in-process, with no serialization or deserialization occurring during function invocations, only during parsing (of Hyperlambda), effectively making it somewhere between 500 million and 2 billion times faster than your message broker.

So if "because it decouples dependencies" is your argument in favor of using microservices, you're literally building a solution that's 2 billion times slower and consumes 2 billion times more time and resources to execute compared to what you could get away with ...

How to Scale Horizontally Then?

Two words; Stateless backends + Kubernetes ...

If I have to explain this in detail, please read a book ...

Conclusion

There are valid arguments for using message brokers, and there are valid arguments for decoupling dependencies. There are even valid points of scaling out horizontally by segregating functionality on to different servers. But if your argument in favor of using microservices is "because it eliminates dependencies," you're either crazy, corrupt through to the bone, or you have absolutely no idea what you're talking about (make your pick!)

Because you can easily achieve the same amount of decoupling using Active Events and Slots, combined with a generic graph object, in-process, and it will execute 2 billion times faster in production than your "microservice solution" ...

"Microservice Architecture" and "Service Oriented Architecture" (SOA) have probably caused more harm to our industry than the financial crisis in 2008 caused to our economy. And the funny thing is, the damage is ongoing because of people repeating mindless superstitious belief systems as if they were the truth.

The reasons for this is because some developers have absolutely no idea what they're doing, or why they're doing what they're doing, and they're instead regurgitating garbage ideas and thoughts they've read about in books some 15 years ago, without applying critical thinking, reason, or logic, in any way whatsoever.

Don't be that guy! Just don't! And if somebody starts saying "microservices eliminates dependencies", suggest decapitation to eliminate headaches and see how they react ...

Or send them my video on PM ...