A Red Pill For (Micro)Services
A Red Pill For (Micro)Services
The unpleasant truth—the red pill—of why we need a new generation of programming languages for dealing with the new machine and microservices.
Join the DZone community and get the full member experience.Join For Free
In my last article here in DZone (https://dzone.com/articles/the-new-computer-machine) I showed how containerization technologies are going to revolutionize the perception we have of a computer machine. In particular, I discussed how we are moving from a computation machine to a coordination machine where microservices represent a key architecture of it. In this article, I will try to explain why we need new programming languages for dealing with the new machine and with microservices.
The reason is very simple and intuitive: usual programming languages like C, Java, Python, etc, were developed for facilitating computation programming, not coordination programming. In these languages indeed, communication and coordination are always achieved by exploiting specific libraries or external frameworks, they are not crystallized within the linguistic structure of the programming language.
Such a fact means that communication and coordination are always treated as a software engineering problem, and they are never approached using the bare constructs of the language. I know, It could appear just as a philosophical issue without any practical consequence, but it is not. There are non-trivial practical consequences because programming is always a matter of knowledge management.
The Management of the Knowledge
Let us consider the case of creating a REST service in Java. Is there a single way for doing it? How many frameworks or libraries could we adopt? Just try to look for "Rest server creation in Java" in a search engine and try to enumerate all the different possibilities with all the possible variants. How many are there? Each of them has benefits and drawbacks. Which one do we select for our next project? Which are the rationales for using the selected solution? I am quite sure that some of the following questions will arise:
- How mature is the library/framework? That is:
- could we safely use it in a production environment?
- do we need someone able to solve low-level issues or could we rely on the available releases?
- Is it documented and how large the community is? That is:
- how hard is it to find a solution when I am stuck with a problem?
- how hard is it to find a developer in the market who knows such a specific framework/library?
- how hard is it to learn the library/framework starting from scratch?
- Is it stable? That is:
- what is the release frequency?
All these questions are related to the ability to manage the knowledge related to the chosen solution. And the knowledge is always related to the skills of the developers. The less knowledge is required, the more developers can use such a solution. The more complicated the solution is, the fewer people can handle it. Thus, the technical choice of a specific solution becomes a team management issue.
The End of Programming Languages
Let us continue with the example of the REST server and let us try to visualize the knowledge required for managing the different parts of the project, we could summarize the scenario in the following simple picture.
It is just a qualitative representation but, as we can see, the knowledge of a pure language (in this example it is Java) represents just a minimal part of all the required knowledge. It is because usually, a programming language varies slowly over time, there can be structural changes for years. On the contrary, communication and coordination libraries are fewer stables, new releases are frequent and new frameworks appear every month. Software architects, engineers, and developers must continuously check and validate new frameworks to safely use them in production environments. These activities take time because they need to build personal knowledge about the new frameworks and the different concepts behind them. This is why in the picture above, the knowledge related to frameworks is represented with the biggest square.
Think to your personal life: once you learned a programming language, how many time do you consult its documentation?
This means that, in a distributed scenario such as that of a microservice architecture, the choice of the programming language is irrelevant. It is more important to choose a technology that facilitates knowledge management. In particular, since every component of the system needs to communicate, a key point is reducing the required knowledge for dealing with communication and coordination. And that's not surprising because, as I said at the beginning of this article, ordinary languages have been built for facilitating computation, not communication and coordination.
You Take the Blue Pill, the Story Ends...
I think all of you remember the scene where Morpheus asks Neo, in the movie Matrix, which pill to swallow. The blue pill just finishes the story of Neo leaving him into the ordinary life thought for all the citizens of Matrix. On the other hand, the red pill allows him to continue his journey into Wonderland.
I don't know if you agree with me about what I explained so far. Maybe you do not, in that case, you could leave a comment for starting a discussion and you could stop reading the article here. But if you think that there is some kind of true and you are interested in investigating some new possibilities that go beyond the ordinary and accepted knowledge about programming languages, you could take the red pill and follow me in the next section.
A New Generation of Programming Languages
In the previous section, I wanted to joke with the quote from the movie Matrix and it is possible that I appeared arrogant. I hope it is not, it was a joke. The example I brought only serves to explain how it is necessary to leap beyond the status quo to understand the revolution that could arise from the advent of the communication machine. In my opinion, the change that is going to come up is so deep that we need to re-think the way we develop and engineer the software
The main idea I am proposing here is that we need a new generation of programming languages for directly dealing with the communication and the coordination aspects to reduce the required knowledge dedicated to them. If we crystallize the basic principles related to the communication and the coordination, we can move the majority of the required knowledge from the area of frameworks and libraries to the area of programming languages.
In this case, we will take advantage of the programming language's characteristic of being stable over a long period to reduce the knowledge needed to tackle a project. As a direct consequence of knowledge reduction, we will be able to reduce development time and reduce maintenance costs. These results can be achieved because the concepts of communication and coordination are directly mapped into language constructs and developers do not have to spend time converting them into the use of a framework or library. As an example, consider talking about classes and objects to an audience of Java developers. All of them will immediately grasp these concepts because there is a direct and concrete correspondence with the syntax of the language.
The same could happen if we talk about services, or synchronous and asynchronous communication, or correlation of messages or, again, compensation of activities and so on, to an audience of developers who know the new programming languages. Young developers will learn the concepts of communication and coordination immediately along with the language, cutting learning time considerably. Legacy technology developers who are struggling to cope with the new challenges of microservices could quickly acquire that minimum amount of knowledge for programming them, simply by learning a new programming language.
Thinking In Services
The contents and the main ideas of this article come from the experience I matured in contributing to the development of Jolie that is a new programming language in the area of services. Started from a research activity at the University of Bologna about modeling Service-Oriented Architecture with process algebras, we noticed that a new language that fully implements a service-oriented paradigm was possible, and we made it.
In Jolie, the unit of programmable software is a service, not an object, not a function, not a procedure, but a service. Such a fact means that the only things a developer can create with Jolie are services or composition of them (that are still services). Here, I do not want to say that Jolie is the new language I discussed in the previous section, but it could be the first example of a new generation of programming languages that can help in clarifying some basic and common concepts they must have. In the following, I present the first list of some concepts which immediately come along with the Jolie programming language:
- Contract first: Jolie is structured in a way that a developer is naturally driven to create the interfaces first. An interface defines the list of synchronous and asynchronous APIs together with all the related message types. It is not necessary to adopt external technologies or languages like swagger, openapi, wsdl, etc for creating a service definition interface, but the language already provides the statements for doing so.
- Binary protocol: Jolie is equipped with a custom binary protocol called sodep that we suggest to use every time we need to connect two or more Jolie services. Such a choice permits to reduce at the minimum the payload of a message. Other protocols can be used like http/soap, http/json, https, etc.
- Structured endpoint definitions: both input listeners and client endpoints (dependencies) must be structurally defined in a Jolie service through the statements inputPort and outputPort. In Jolie a service can receive and send messages, thus it can act both as a server and a client at the same time. Moreover, different protocols can be defined for each endpoint, thus the same service can receive or send messages using sodep, using http/soap, or as a rest service using http/json.
- Behavior as a workflow: in Jolie, the behavior of each API takes the form of a workflow. Besides the usual sequential operator, Jolie is equipped with a parallel operator which easily allows for composing activities in parallel.
- Tree node vectors values: in Jolie, all the variables are tree nodes of vectors. Similarly, as a JSON value, each variable can contain a value and/or a set of subnodes that are vectors of one or more elements. The syntax permits to navigate nodes easily.Compensation, termination, and fault handling: Jolie offers a rich set of possibilities for managing fault scenarios. Besides the well-known mechanism of fault handling, Jolie also provides primitives for managing compensations and termination handlers when more activities are programmed in parallel.
- Running more services into the same cell: thanks to the embedding primitive it is possible to execute more than one service in the same execution environment that we call cell. Since so far the only available engine for Jolie is developed in Java, a cell corresponds to a Java Virtual Machine.
- Correlation sets: Jolie provides a specific mechanism for managing message correlation sets which permits to correlate past messages with the same session.
- Architectural primitives: Jolie offers the possibility to architecturally compose the endpoints using three main primitives: aggregation and redirection. In the former case, more services can be exhibited within the same input port of a master service called aggregator. We call surface, the resulting aggregated interface at the aggregator input port. In the case of the redirection, the service plays the role of a proxy.
- Courier definitions: in the context of an aggregation, Jolie offers a specific primitive, called courier, for analyzing incoming messages before forwarding them to the final target.
This is just a list of some of the basic features offered by Jolie. Maybe they do not represent a novelty by themselves, but it is certainly interesting to have all of them embedded in a unique linguistic tool. In particular, the combination of them allows us to address microservice programming differently. As an example, in my first article in DZone I explained how it is possible to approach a Jolie project without taking care of the question: do I develop a monolith or a distributed application? Such a result comes from the fact that in Jolie everything is a service and there is no difference between programming a monolith or a distributed set of services. Jolie is an open-source still evolving project whose engine is developed in Java. It has been tested in production environments where it has been used with success.
In this article, I tried to explain how a new generation of programming languages is needed for dealing with containerization technologies and microservice programming. The main driver is the reduction of the communication and coordination knowledge which is at the basis of each distributed system project. Starting from our experience with Jolie, we can say that it is possible to program using a service-oriented paradigm and that it can help in reducing development and maintenance costs. The community around Jolie is still small but we believe it could grow if we can describe all the novelties it introduces. Creating and introducing a new programming language is a challenging task, but we took the red pill some years ago and we are learning step by step how to manage it. People who are interested in helping the project to grow can contact us, we will be happy to introduce them to the Jolie community!
Jolie was not born in a garage but it emerged from a process calculus we developed some years ago. At that time we developed a three-layer operational semantics for dealing with different aspects of service-oriented computing. Each of them has been mapped into Jolie. From that day Jolie has been evolved enormously, but such a calculus is still a valid reference for theoretically approaching the semantics of Jolie.
Opinions expressed by DZone contributors are their own.