Automated Microservices Observability in the Cloud
Let's take a look at how we can achieve automated microservices observability for our on-premise applications via the cloud.
Join the DZone community and get the full member experience.Join For Free
Observability is a critical aspect to consider when building a complex application. You need to constantly have visibility about the application’s status in order to know its current health and predict its future state. There are many tools and technologies to implement observability, such as Prometheus for metric collections, OpenTelemetry API-based implementations for distributed tracing, and numerous log processing systems. However, all of these systems have to be configured and deployed by someone. Moreover, your application needs to be updated suitably to emit the information required to be sent to these systems.
But what if we don’t have to do any of these? What if we simply have to implement our applications and all the observability aspects would be taken care of? This is what the Ballerina programming language and the Choreo integration platform as a service (iPaaS) have done.
Ballerina provides the ability to observe your code without any explicit code-level intervention by automatically observing operations that are most often observed. This is done with unique language features: first-class support for network interactions in the language such as services, client endpoints, and remote functions. Ballerina supports a pluggable model for integrating with many observability platforms, including Prometheus and OpenTelemtry, and it also has built-in support for Choreo.
By using Choreo, any existing Ballerina application can utilize the platform to publish the observability information, which will be summarized and visualized in the cloud. By performing a single configuration update, any Ballerina application that is deployed in your local data center, or even your personal laptop, can be observed using Choreo. This feature is known as Remote Apps in Choreo.
Next, we will go through the steps on how to configure an existing Ballerina application as a Remote App in Choreo.
Microservices Use Case: An E-Commerce Backend
We will be using an e-commerce backend that simulates the services required to implement searching for goods, adding them to a shopping cart, doing payments, and shipping. The solution does not implement all the features of a real-world implementation, but rather follows the simple patterns and functionality you would be faced with.
The services are exposed through HTTP. We will also be using MySQL as the persistence mechanism.
Figure 1 shows the overall architecture of how the services interact with each other.
For each main domain, we have a microservice to handle the work such as searching the current inventory for goods, storing items in the shopping cart, and creating orders with the current items in the shopping cart. A single “Admin” service fronts these services to create a unified interface for the website to interact with the backend. So, the “Admin” service will be the main orchestrator in connecting the backend services.
Configuring Choreo Remote Apps
In order to deploy our application as a Choreo Remote App, we need to enable observability in our application. This is done by setting the configuration property “ballerina.observe.enabled” to “true”. This can be set by adding a “Config.toml” file to our Ballerina package’s root directory and setting the content similar to the following.
[ballerina.observe] enabled=true provider="choreo"
Afterward, we have to import the “ballerinax/choreo” module in our Ballerina package in order for the Choreo extension to be bundled into our final executable. This can be done by adding an import statement similar to the following:
import ballerinax/choreo as _;
In our e-commerce sample application, we would create “Config.toml” files in the services and the simulator Ballerina packages. Afterward, the “ballerinax/choreo” are added in the admin.bal and the simulator.bal files in order to signal the compiler to bundle the Choreo extension in the final executables.
Build and Sample Run
$ bal build Compiling source laf/ecommerce:0.1.0 Generating executable target/bin/ecommerce.jar $ bal run target/bin/ecommerce.jar ballerina: initializing connection with observability backend periscope.choreo.dev:443 ballerina: visit http://console.choreo.dev/observe/app/db278430-094b-11ec-bd7d-ee59efa67041/db279662-094b-11ec-bd7d-ee59efa67041 to access observability data ballerina: started publishing metrics to Choreo [ballerina/http] started HTTP/WS listener 0.0.0.0:8082 [ballerina/http] started HTTP/WS listener 0.0.0.0:8084 [ballerina/http] started HTTP/WS listener 0.0.0.0:8085 [ballerina/http] started HTTP/WS listener 0.0.0.0:8083 [ballerina/http] started HTTP/WS listener 0.0.0.0:8080 [ballerina/http] started HTTP/WS listener 0.0.0.0:8081
$ bal run target/bin/simulator.jar -- 100 100 ballerina: initializing connection with observability backend periscope.choreo.dev:443 ballerina: visit http://console.choreo.dev/observe/app/16dd8938-094c-11ec-bd7d-ee59efa67041/16dd99a0-094c-11ec-bd7d-ee59efa67041 to access observability data ballerina: started publishing metrics to Choreo ballerina: started publishing traces to Choreo
Here, in the first build and run sequence, the e-commerce backend services are run. This consists of total size services which are run at once. At runtime, it communicates with Choreo and generates an URL that can be used to access the observability view of the deployed services.
In the same manner, the simulator package is built and run, which sends requests to the “Admin” service in order to search the inventory, add items to a shopping cart session of a specific user, and then finally checks out the items. These service interactions are also observed and can be visualized in its own dashboard provided by Choreo.
Below shows the live observability views provided by Choreo after the Ballerina applications are deployed and running.
Admin Service Inventory Search.
Admin Service Checkout.
Admin Service Add to Cart.
Billing Service Make Payment
Shipping Cart Service Add Item
In this article, we looked at how we can easily engage observability aspects to on-premises applications and offload the analytics and the visualization aspects to a solution in the cloud through Choreo.
Published at DZone with permission of Anjana Fernando. See the original article here.
Opinions expressed by DZone contributors are their own.