How to Use the AppMon Business Transaction Feed to Export Data to a Play Microservice

How to Use the AppMon Business Transaction Feed to Export Data to a Play Microservice

My name is Chris André and I have been a Dynatrace AppMon expert for years. So much so that I sometimes feel as if this tool was my own. Unfortunately, being so used to it, I can't help but feel bad when I see that my customers don’t use it to its full extent. Think of it as having a Ferrari and only using first gear: it’s nice and everybody envies you, but you could do so much more.

This blog post is my contribution to those users who want to take their AppMon skills to the next level. In it, I give a step-by-step guide for using a powerful (but relatively unknown) AppMon feature: the real-time Business Transactions Feed.

In this case, I'll combine AppMon and its out-of-the-box features with a micro-services framework I have been using for years: Play from Lightbend.¹ This framework is often used in microservice architectures as it has multiple benefits: extremely small and easy to deploy, non-blocking IO, low memory usage, handles JSON natively, etc. It is also built on top of the Scala programming language which is used more and more for large in-JVM data manipulation (most notably thanks to the Apache Spark project!).

Why This Article?

AppMon is a fantastic tool that gives you a wealth of information. However, it doesn't take long to realize that beyond raw performance data, there is valuable business information locked inside. This article shows you how to unlock that data and use it in your daily reporting. Examples of what you can do with it are numerous and the sky is your limit!

• You could combine the results of business transactions and/or bypass AppMon's technical limitations (in particular, splitting limits). As Shrimant, a former colleague (and current AppMon administrator), told me:

  "...Storing historical data with AppMon is difficult, and doesn't allow you to keep high resolution data for problem solving. This is a critical function for organizations with slow and cumbersome mechanics. While one could constantly keep and store session exports, there is far less tax on network and server architecture if one uses a live BT feed into a solution designed to process large amounts of data."

• You could use AppMon as a datasource for your loyalty offer system: this way if one of your most valued customers encounters a problem while making a purchase, you could proactively call him and help him complete his purchase. 
• In my case, I decided to use the exported data in an OLAP datawarehouse using Mondrian. This would allow me to slice and dice the captured data and run more detailed analysis on what is happening on my website. Though it is not covered in this post, this might be the subject of another one.

Today, to remain as simple as possible, I will focus on the data export. While doing it, I'll avoid platform specifics so everyone can understand what I am doing and adapt it to their system.

AppMon Configuration

In this part, you will:

• Learn what a Business Transaction is.
• Learn where the server-wide configuration is located.
• Learn how to specify which Business Transaction should send its data.

Description of What Business Transactions Are an AppMon Server Configuration

Business Transactions (sometimes shortened as "BT") are a very powerful feature of AppMon. Think of them as queries applied to the transactions your system processes. These queries allow you to extract business data from elements of your application (parameters, method arguments, return values, etc.) and create charts or execute deep analysis of your application.

Thanks to this, you can answer questions such as "how many web requests have a response time longer than 1 second?" or "give me a list of all users who logged in more than once in the past 10 minutes."

This is a fascinating subject and you can find a wealth of details online (For example, the official documentation or this blog post from Andi Grabner). For now, we'll focus on how to export data from an existing Business Transaction: in this case, we'll reuse an existing demo system and print some of its data.

How Does AppMon's Business Transaction Feed Work?

In simple terms, when you activate the Business Transaction feed, the AppMon server sends data to a remote URL. It uses the Protobuf protocol from Google to serialize it in a POST request. It is then up to the receiving component to process this data. For example, it can save it on a high-volume database or on an HDFS storage for further processing. You can find more details about this on the official Dynatrace documentation!

Configure the AppMon Server to Send Data to the Play URL

Let’s start this tutorial by configuring the AppMon server to let it know where to send the Business Transaction data.

We'll do this in the server settings window. 

Once the new window is open, select the “Realtime Streaming” tab and then the “Business Transactions Feed” sub-tab.

On this window, we activate the Business Transaction Feed by selecting the checkbox at the top and filling in the various fields. As you can see in my screenshot, I have only configured two of these fields:

• The first one, “URL”, is the URL where my AppMon server should send the BT data. Here, I have configured it to send the data to my development box on port 9000. This port is the default port on which Play applications listen and is, of course, configurable. I have also defined the URI as “/BtExport”: we’ll use it later when we configure Play.
• The second one, “Bulk size”, indicates the maximum data entries I will send at one time. If I don’t reach this limit, a one-minute timeout will flush data captured within the past minute.

Other visible fields are security-related, so we'll ignore them here. Note, however, that you may need to configure them in your environment.

Configure the Business Transaction to Send Captured Data to the Remote System

Now that the server configuration is complete, let’s select the Business Transaction whose results we want to export.

Here, I will use a Business Transaction from the demo system profile “easyTravel” called “easyTravel Logins by Username.” This Business Transaction tells us who is logging into the application.

We open the details of the Business Transaction by right-clicking on its name. Then, we select the “Edit Business Transaction” option on the pop-up menu.

Once on this screen, the only modification we need to do is to check the “export results via HTTP” button. This will send the captured data to the URL we configured in the previous step.

The two other checkboxes are used to increase the details included in the data.

• “PurePath references” allows you to drill-down back to the PurePath from which the data comes.
• “Performance data” allows the export of performance-related metrics from the PurePath (CPU time or suspension time, for example).

How to Create a Scala Play Project

We have configured AppMon: now is the time to set up the application that will consume the data.

In this part, you will:

• Install Activator.
• Create a basic Play project.
• Modify the basic Play project to accept incoming POST requests.
• Learn how to launch the Play project in dev mode.
• Test if your application can accept POST requests.

Get Activator Installed on the System

Activator is a tool provided by Lightbend, the editor supporting Play commercially. It contains many different templates to kick-start your Play deployment.

To install Activator, you need to go through 3 steps:

• Download the zip from its website (https://playframework.com/download#activator).
• Extract the zip into a dedicated directory.
• Edit the PATH variable of your environment to include the location of the “bin”subfolder. On MacOS, you can edit the file “.bash_profile” located in your home folder. For all other systems, you can follow instructions found here:.

Be aware that the first time you launch activator, it will attempt to download the different libraries needed to launch. Please make sure you have an internet connection then.

Run the Seeds

Activator provides you with templates that illustrate how to integrate various technologies. It also provides you with the basic skeleton of a Play application (called a "seed"). Now that it is installed, you can create a new play application by going into a new folder where you want to create your application and typing:

    activator new DynatraceBtExport play-java

Where ‘DynatraceBtExport’ is the name of your Play application. This will create all files required for a basic Play application. You can make sure it was all installed properly by then typing:

    cd DynatraceBtExport/
    activator run

This will launch your Play application, letting you confirm that it all starts properly. Once you are done with it, you can then press Ctrl + D to stop the application from running.

Change the Project to Handle Incoming Requests

At this point, we've created the skeleton of our project: let's edit it so that it can handle the incoming requests.

First, We Need to Create (or Reuse) a Method Contained in a Controller to Handle Incoming Requests.

Play has Controller classes used to process incoming requests. These classes have various methods that return Action objects describing the type of response to send back to the browser. For example, an Ok object is similar to an HTTP 200 response, an InternalServerError object maps to an HTTP 500 response.

Here, Activator generated some methods within the class controllers.HomeController for us in the last step. Since we haven't implemented any logic yet, let's reuse the method index to process the requests sent by the remote server. Our previous step generated this method and, in order to make sure we execute our code, we'll change the code to something more customized ("hello DT world" anyone?).

Then, We Need to Tell Play That We Want a Specific URL to be the Dynatrace Server's Target.

To do that, we need to edit the file routes usually located under <project root>/conf. This file is the routing table of the incoming web requests and 3 parts divide it:

• The HTTP method used by the request (GET, POST, PUT, etc.).
• The URI of the request.
• The associated method in the Controller object that handles this request.

In our case, the AppMon server will send us a POST request with a body that contains the serialized objects. Thus, I need to put in a new entry as follows:

    POST    /btExport        controllers.HomeController.index

This maps the method index from my Controller class to all POST requests addressed to the specified URI.

Run the Application in Dev Mode to Test It.

Now that we defined the new route, it is time for us to launch the application and make sure it works as expected. Fortunately for us, Play provides us with a development mode that allows us to test our modifications on the fly. The easiest way to launch it is to open the command line prompt and type the following command:

     activator run

This launches Activator in development mode. By default, it'll listen for incoming requests on port 9000. Let's give it a try!

Since the HTTP method we want to try is POST, we cannot just type the address in our browser as it would send it as a GET. Instead, we'll type a command in the command line to execute this web request:

    curl "http://localhost:9000/btExport" -X POST

This command works on Linux and on OS X and will display the result message on the stdout.

For Windows, you will be able to use it if you installed Cygwin on your machine. Otherwise, you might have to install a more specific tool (such as Postman) to run this test.

Protobuf Usage

In this part, you will:

• Learn what protobuf is.
• Import its library into our Play project.
• Retrieve the ".proto" descriptor file from AppMon.
• Convert a ".proto" file into a Java class.

Import protobuf as a Library for the Activator Project

As mentioned before, the AppMon server posts its data using Google's protobuf format (AKA "Protocol Buffers").
Dynatrace chose this binary format for its platform-neutrality combined with its optimized speed and small bandwidth usage. As mentioned on the "Why not use XML?" part of the Protobuf website, this format is: 

• Simpler.
• 3 to 10 times smaller.
• 20 to 100 times faster.
• Less ambiguous.
• Generates data access classes that are easier to use programmatically.

The server serializes a series of objects with the results of the Business Transaction using this protocol. The result of this serialization is then put into the body of a POST request sent to our application server.

To add the library to the CLASSPATH, the easiest way is to edit the 'build.sbt' file located at the root of your project and add the following line:

    libraryDependencies += "com.google.protobuf" % "protobuf-java" % "3.1.0"

This file, automatically created by Activator, is the project deployment descriptor. Modifying it will tell Activator to import the library into the CLASSPATH and use it.

Generate a Java File from the protobuf File Provided by Dynatrace

Obtaining the ".proto" File

A ".proto" file describes the objects' format (like a WSDL in the web services world). You can download this file by clicking on the hyperlink in the "Business Transaction feed" window.

When you click, a new window will appear: click on the "save to file" button and save the file somewhere on your disk.

Install the Protobuf Compiler

Now that we have the file, we can generate the Java file that we'll use to deserialize the data. To do so, you first need to install the protobuf compiler: this program will convert the ".proto" file into its Java translation.

I will not get into the details of the installation process, but you can download the installer binary on GitHub (https://github.com/google/protobuf).

Generate the Java Source File

Once the compiler is installed, you can test the installation by typing the following in the command line:

    protoc --version

This will return the version of the protobuf compiler you have installed on your system. If everything works fine, you now can generate the Java file by typing the following command:

    protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/BtExport.proto

Where: * $SRC_DIR is the location of your source. * $DST_DIR is the directory where the compiler will generate the Java source files. * BtExport.proto is the name of the proto file I retrieved in the previous step.

In my case, since I was in the same folder as my ".proto" file, I typed:

    protoc -I=./ --java_out=./app BtExport.proto

Once the compiler completes, your source files will be located in the output folder.

Use the Object to Read Data When We Receive It

In this part, you will:

• Capture the body of a POST request.
• Deserialize the content of the request into a Java object.
• Parse the Java object to display the data it contains.

We now have all the elements necessary to work with the data sent by the AppMon server. Let’s do something with it. Since the purpose of this post is to test the data export, we’ll keep it simple and only print the data to the standard error output. Here is how we’ll do it: * First of all, we retrieve the data from the request’s body and deserialize it.

    byte[] byteArray = request().body().asBytes().toArray();
    BtExport.BusinessTransactions bts = BtExport.BusinessTransactions.parseFrom(byteArray);

To do so, I first retrieve the body of my request as a bytes array and parse it using the parseFrom method provided by the class BtExport.BusinessTransactions.
This class was created automatically when we generated the Java sources from the ".proto" file.

• The deserialization returns a BusinessTransactions object that has an underlying list of BusinessTransaction objects (notice the missing S at the end?). Each BusinessTransaction object is itself composed of many Occurrence objects.

The BusinessTransaction list is the list of all Business Transactions configured to send data through the interface. For each object, you can identify it by using the getName() method that returns its name. Within each BusinessTransaction, we have a list of Occurrence objects which represents all PurePaths matching the Business Transaction. Here, we will get the response time (obtained by calling getResponseTime) as well as the name of the associated user.

This last bit of information is returned as the first “split measure” defined for the Business Transaction. Here, it translates as the first dimension of the “dimensions” attached to the occurrence.

    List<BtExport.BusinessTransaction> businessTransactionList = bts.getBusinessTransactionsList();
            for (BtExport.BusinessTransaction bt : businessTransactionList) {
                for (BtExport.BtOccurrence occ : bt.getOccurrencesList()) {
                    String message = String.format("%s - %s took %.3f ms to log in", bt.getName(), occ.getDimensions(0), occ.getResponseTime());

                    System.err.println(message);
                }
            }

Conclusion

As you can see, there is no black magic in exporting data to a remote system using the “Business Transaction feed” feature from Dynatrace. As a matter of fact, it was remarkable how straightforward this was: in less than half a day of research, I was able to get the first version of this post running. If you want to know more, I have posted the project on GitHub (which can be found here). If there are any questions or comments, please feel free to let me know and it’ll be my pleasure to answer you!

1. I could have used other frameworks and tools (NodeJS, Elastic Search...) but I know Play better and decided to go with a tool I work with every day.

2. Play developers may notice that I am not using Play the way I should. I am perfectly aware of this but, for simplicity's sake, I decided to remain as basic as possible.

The source and most up-to-date version of this article can be found at here.: https://github.com/EllanVanninCA/DynatraceBtExport

The whole content of this article as well as its associated source files are under the Apache 2 License.