July 2017: the stage is set for a new dancer to come pirouetting onto the integration scene. Ballerina, an open-source language, editor, and above all, a new integration approach, is introduced by WSO2. It is precisely what Ballerina does not do as much as what it does do that makes this new entry so interesting and potentially compelling for companies, large and small.
Why a New Integration Solution?
Ironically, the underlying monolithic architecture of ESB servers causes them to suffer from the same issues as the integration problems they set out to solve.
On the operations front, the fact that installations have traditionally favored hub-and-spoke topologies create problems that many first-time integrators do not expect, in particular, inflexible deployment and un-deployment due to versioning, library management, stringent downtime constraints, and other dependency issues. For developers, traditional integration solutions often come coupled with an unwieldy development language, missing or limited namespace support, a poor, unsupported testing framework, or none at all, lack of typesafety, no compiler step, and source-control-unfriendly visual editors that are chronically corrupted or out of sync vis-à-vis the actual source.
Integrators end up being surprised and frustrated with their ESB tooling precisely because the problems are so unexpected and so utterly contrary to the marketing message of the vendor: integration solutions are supposed to address environment complexity, not create it. The solutions are simply not as mature as what developers are used to in the Java space, both for operations and development.
Can Ballerina help? Let’s take a look!
Each Solution Is Stand-Alone
Ballerina avoids the problems of monolithic architecture altogether, in that it simply turned down the invitation to come to the party. Ballerina started her own dance, one in which each integration solution is a stand-alone solution running in a Docker container. You can bring it up or take it down at will without affecting any other integration solution you might have running in your enterprise.
What If You Need More?
Ballerina is not a replacement for Java, or for any other mainstream programming language for that matter. You could implement business logic in Ballerina, but you probably won’t want to. With Ballerina you don’t have interfaces, classes, dependency injection, sophisticated testing frameworks, established best practices and experience with your CI/CD infrastructure and so on. As a result of this, you will be far less likely to to succumb to the temptation of allowing business logic leakage into your integration solution. What you probably will prefer, and what somehow comes naturally, is to use Ballerina to achieve decoupling: doing a bit of message transformation here, maybe some routing there, some aggregation here. Then Ballerina exits stage left.
If you need more than that, create a service in Java, Kotlin, or any other language you normally develop in, hosted somewhere else, say in a Docker container, and call it from your Ballerina container instance. To achieve an easily deployable encapsulated end-to-end solution, you can package all of the components required for a solution with docker-compose.
Ballerina offers a browser-based visual code editor which is surprisingly user-friendly. It is a great first stop for getting to know the language, because the core components of the language are presented as icons which can be dragged onto a technical-drawing-paper-motif grid. From there, you can switch to Code View to see what you just did. The other way around, writing code first, works similarly. This will not appear strange to not-so-old-school programmers, those who know Delphi, Swing or JavaFX. It is a handy way to become acquainted with the language. Like the programmers in The Matrix, however, you may find yourself quickly graduating to a code-only approach: the language is highly readable.
Ballerina is based on the concept of a sequence diagram. This is not so very different from what ESB products have done in the past, but now with a vertical data flow depiction instead of one that used to be horizontal. But this innocuous change has powerful consequences beyond simply making better use of screen real estate: it becomes easier to conceptualize the interaction of processes, which may or may not be sequential in nature.
Code and Visual Editor Always in Sync
In traditional ESB products, there is often a custom file per project which controls the appearance as well as the layout of the visual representation of the solution. These files tend to get out of sync with the actual integration code. This causes quite a few headaches for developers trying to re-establish control of a project that just went haywire. In Ballerina, the visual representation and the actual code is one and the same — there is no intermediate file. If you change the code, you change the visual sequence diagram, and vice-versa. There is literally no file that can get out of sync or corrupted. Since the visual solution depiction is code itself, that too is source-control friendly and serves as supplemental visual documentation, which can be very valuable as a communication medium, particularly when interacting with non-programmer stakeholders.
The elimination of project-specific configuration files probably did not hurt in enabling the Ballerina authors to more easily support popular IDEs such as IntelliJ IDEA, Eclipse, Visual Code and Atom. Cursory testing on IntelliJ indicated that code formatting, code coloring, and code navigation worked as advertised, but the level of sophistication is not that what you will be used to with say Java or Kotlin. There was no issue in switching back and forth between IntelliJ and the Ballerina Composer. It was a fairly workable environment.
An overview of all of the features of the language is better provided by the Ballerina documentation itself, links below; however, a few features merit mention.
No More XML Ad-Infinitum
Pages and pages of XML in established ESB products had programmers gagging. On the other hand, it was portable. Sheesh. It is hard to overstate how unworkable XML really is for complex solutions. It is akin to asking your people to get good at Klingon.
Ballerina has chucked all of that silliness over starboard. Instead, it favors a functional, typesafe language with readable flow control, error handling with compile-time checking.
The key concepts of Ballerina are Services and Resources, Workers, Connectors and Actions, Functions, and Annotations. The Ballerina concept documentation has excellent explanations of all of these. Again you can see from the names of these core concepts alone how the language is focused primarily on integration.
XML, JSON, and data tables are all there as first-class citizens in the language. Beyond that, there are structs which can be used as data containers as well.
Common Integration Abstractions and APIs
Mime, HTTP, SQL-D, WebSocket, FTP, File and many others have first-class representations in the language. Common APIs such as for FaceBook Gmail, LinkedIn and others are provided out of the box.
Java programmers will feel right at home with the try/catch/finally concepts of Ballerina. In addition, you have the ability to return multiple return values which give you added flexibility.
The Ballerina language has parallelism built in. The sequence diagram approach makes it easier to reason about and visualize parallel processes.
main() or Service, Take Your Pick
It is trivial to package up a solution as a functional program with a main() entry point or as a service, providing multiple network-accessible entry points.
In addition to its self-documenting nature given the fact that a Ballerina solution is at the same time a sequence diagram, Ballerina offers the ability to document itself as a Swagger file. Swagger definitions can also be imported.
Ballerina and Swagger YAML and graphic syntaxes are interchangeable.
Ballerina is a cross-platform language that is compiled on the fly into in-memory byte-code during the Run operation. Future versions will persist the bytecode to disk, separating Compile and Run into two distinct steps. Ballerina runs on the JVM, but that is an implementation detail which could change, for instance to LLVM. Even so, the Ballerina authors are committed to the language remaining cross-platform. I have performed cursory testing of many of the samples on a Mac, on Ubuntu Linux, and on Windows 10 and they worked without any change, other than path modifications for Windows.
Ballerina favors a micro-architecture packaging and deployment approach using Docker. A single integration solution can be entirely self-contained. Artifact reuse can be accomplished via various approaches: You can create a Ballerina service with multiple service entry points, you can use docker compose, you can call functions in other standalone containers, you can embed functionality as libraries. At this time, there is, however, no maven-like build file that can import dependencies for you.
Fast Startup Time
Obviously, startup time will depend on what is put into a container, but Ballerina containers start up super fast, typically in < 1 - 2 seconds.
As with all things, you can come up with architectures that suppress the stated goals such as single-purpose, stand-alone integration solutions packaged in a single container or in a container composition (docker-compose). But with Ballerina it does get harder to string yourself up.
Injection of environmental variables is delegated to Docker at the moment, and hopefully the Ballerina architects will not sense the urge to fix it.
Logging and Monitoring
Here too, Ballerina outsources its own log aggregation and monitoring functions to dedicated services that are typically found in cloud or container-supporting environments, or to services that you write yourself for that purpose.
Supporting Cast: Documenting, Testing, and Packaging
No self-respecting Ballerina would think of a solo performance without music, without other dancers, without lights or stage equipment. For this purpose, Ballerina brings to the stage a supporting cast for documentation, testing and packaging, named, unsurprisingly, Docerina, Testerina, and Packerina.
Alternatives and Substitutes
You do not need a dedicated integration language to do integration. Java EE and .NET, to mention two major players, have APIs that specifically address integration concerns (web services, batch, timers, database access, and so on). Then there are integration libraries such as Camel that have large communities and mindshare and which have been proven to be very effective.
Ballerina architects state that if 80% of your service is about integrating with other services and data, then Ballerina is a great option. If only 20% pertains to integration, then use something else. The use of container technology has made the choice for an integration approach far less of a one-size-fits-all proposition.
As far as substitutes, either you integrate or you don’t. But no company has unlimited resources. You may decide that single point-to-point situations or one-offs do not merit a decoupled integration due to missing reuse potential. Other factors may weigh in as to when and what you decide to integrate. Obviously, asynchronous needs lend themselves to decoupled solutions more so than do synchronous communication models.
License and Community
Ballerina is licensed under Apache License 2.0, making it attractive for both open-source and commercial usage.
The community is very open and active. All development is open (see resources, below).
Maturity of Language
The bug tracker still has quite a few bugs, and a first 1.0 version is not yet out. The language spec is still in the process of settling down. As such, it would be premature at this date to recommend it other than for exploratory purposes. Both the open-source nature of the project along with the backing of a major integration company and its clear commitment to the language do provide a reason for confidence that the approach that Ballerina portends will, in fact, be viable long-term.
You can get involved in the following ways:
Other Links of Interest
Any opinions expressed are my own.