Serverless Integration and Distributed Tracing in Kumologica

Distributed tracing is a key aspect in the new world of serverless integration as it is one of the 3 pillars of observability i.e logs, metrics, and traces. Distributed tracing is often considered hard to implement in many enterprises because of multiple reasons as it comprises of several disparate components such as:

1. Instrumentation of service.
2. Context propagation.
3. Trace ingest.
4. Trace storage.
5. Trace retrieval and visualization.

Having a distributed tracing infrastructure is one part of the equation whereas instrumenting the application to enable trace ingestion is different from a set of tasks.

In this article, I am going to show you how a distributed tracing can be achieved in serverless integration with zero instrumentation and configuration. For this, we will be using X-Ray service from AWS as the distributed tracing infrastructure and Kumologica for building the service that will be deployed as a Lambda.

AWS X-Ray would solve the infrastructure part by providing the capability to store, retrieve, and visualization of service graph but it doesn’t solve the pain point of implementing the instrumentation in your service. Though AWS gives the necessary instrumentation library to ingest the trace to X-Ray the developer still needs to understand the X-Ray SDK to apply the necessary instrumentation in his service. This is an extra effort for developers to put on top of building the actual business functionality.

Developers need not worry about this anymore as this problem is solved in Kumologica. Services that are developed on Kumologica doesn’t require any explicit instrumentation as it is taken care of by the underlying Kumologica runtime library. Every node used in Kumologica is by default instrumented to ingest the trace data. Based on the enablement of X-Ray for your service, the trace data will be ingested to the AWS X-Ray service.

For the demonstration, I have a simple Kumologica service flow as shown below which I will be deploying as AWS lambda in my account.

Building the Flow

Before the building, the flow ensures that you have the latest Kumologica Designer downloaded and installed in your machine. If you are not familiar with Kumologica then I would recommend to go through the following article or watch our youtube channel. Now let’s start.

1. Add the EventListener node to the designer canvas from the palette and configure the following on the node settings.

Event Source: Amazon API Gateway
Verb: GET
URL: /xray

2. Add the logger node and set the Message as “Request received”. Wire the EventListener node with logger node. The log message will be captured in the AWS Cloudwatch service.

3. Add HTTP Req node to the canvas and configure the following. Wire the logger node with the HTTP Req node. This is to invoke an external service.

Method: GET
Path: /xray
URL:  http://httpbin.org/ip

4. Finally we will add the EventListener End node and configure the following.

Payload : msg.payload

5. Wire the HTTP Req node with the EventListener End node.

Deploying and Enabling X-Ray

1. Go to Cloud Tab in the designer and under the Trigger section add Amazon API Gateway as the trigger.

2. Expand the More Parameters section and enable the Activate X-Ray checkbox.

Activate X-Ray configuration will enable each node to emit trace details into the AWS X-Ray service.

3. Click Deploy.

Try it

1. Invoke the endpoint URL that you received in the Kumologica Designer terminal. eg: https://<<gateway instanceid>>.execute-api.ap-southeast-2.amazonaws.com/test/Xray

2. Go to your AWS account. and search for the X-Ray service.

3. Click on the traces and you will find the following service graph along with trace data details reporting the execution time of each node, external service, and overall flow execution.

Above is the service graph which gives the relationship between the lambda functions and the external systems integrated.

As you can see from the above trace details that each node and its execution time is being captured on X-Ray without the developer doing any extra effort on X-Ray instrumentation.

Summary

This article has shown how easy to enable distributed tracing with AWS X-Ray on a service developed using Kumologica Designer.