Telegraf, which is part of the TICK Stack, is a plugin-driven server agent for collecting and reporting metrics. It has plugins or integrations to source a variety of metrics directly from the system it’s running on, third-party APIs or even other services like StatsD and Kafka consumers. It also has output plugins to send metrics to a variety of other datastores, services, and message queues (not restricted then to InfluxDB only). In this article I am going to describe the setup to pull metrics from Java applications hosted inside a Docker container. The release of Telegraf I am referring to here is the 1.3 (the latest at the time this article is written). I am also assuming you have the Telegraf agent already installed in the node where the Docker container is going to be shipped.
Telegraf Configuration to Pull Metrics From a Docker Container
Telegraf comes with a plugin to pull metrics from Docker containers. The following is the typical setup you have to specify in your Telegraf agent configuration file (which is /etc/telegraf/telegraf.conf for Linux hosts):
[[inputs.docker]] ## Docker Endpoint endpoint = "unix:///var/run/docker.sock" ## Only collect metrics for these containers, collect all if empty container_names =  ## Timeout for docker list, info, and stats commands timeout = "10s" ## Whether to report for each container per-device blkio (8:0, 8:1...) and ## network (eth0, eth1, ...) stats or not perdevice = true ## Whether to report for each container total blkio and network stats or not total = false ## docker labels to include and exclude as tags. Globs accepted. ## Note that an empty array for both will include all labels as tags docker_label_include =  docker_label_exclude = 
Please have a look at the Github readme file for this plugin for the full list of metrics you can pull from any Docker container. Those metrics give you a full picture of what's going on for the containers, but no specific info about the Java application running inside. For this reason, you have to configure a second Telegraf plugin.
Telegraf Configuration for Jolokia
In order to pull metrics from any application running on a JVM, Telegraf comes with a specific plugin which uses the Jolokia agent. Jolokia is an agent based approach for remote JMX access. The communication between client and agent goes over HTTP (either GET or POST), where the request and response payload is represented in JSON. The Jolokia plugin collects JVM metrics exposed as MBean's attributes through a Jolokia REST endpoint. In the Telegraf configuration, you have to setup the Jolokia endpoint details for the input plugin:
# Read JMX metrics through Jolokia [[inputs.jolokia]] ## This is the context root used to compose the jolokia url ## NOTE that Jolokia requires a trailing slash at the end of the context root context = "/jolokia/" ## List of servers exposing jolokia read service [[inputs.jolokia.servers]] name = "as-server-01" host = "127.0.0.1" port = "8778"
and the metrics you need to collect. Here's an example:
## List of metrics collected on above servers ## Each metric consists in a name, a jmx path and either ## a pass or drop slice attribute. ## This collects all heap memory usage metrics. [[inputs.jolokia.metrics]] name = "heap_memory_usage" mbean = "java.lang:type=Memory" attribute = "HeapMemoryUsage" ## This collects thread counts metrics. [[inputs.jolokia.metrics]] name = "thread_count" mbean = "java.lang:type=Threading" attribute = "TotalStartedThreadCount,ThreadCount,DaemonThreadCount,PeakThreadCount" ## This collects number of class loaded/unloaded counts metrics. [[inputs.jolokia.metrics]] name = "class_count" mbean = "java.lang:type=ClassLoading" attribute = "LoadedClassCount,UnloadedClassCount,TotalLoadedClassCount"
At this stage, the Telegraf configuration is ready. You need to restart the agent to make it effective. One thing is still missing for this process: a running Jolokia agent listening at the port specified in the plugin configuration and attached to the Java application process.
Jolokia Agent Configuration for the Java Application Inside the Docker Container
The Jolokia agent can be shipped in the same Docker container for the Java application to be monitored. You need to add some instructions to the Dockerfile for your Java application.
You need to add the Jolokia JVM agent to the image:
ADD /<local_path>/jolokia-jvm-<version>-agent.jar /
Then expose the agent port:
and finally provide the following single startup option to install the agent when the Java process starts:
CMD ["<other_options", "-javaagent:jolokia-jvm-<version>-agent.jar=port=8778,host=0.0.0.0"]
Build your image as usual and then ship and start a container in the destination machine where you started the Telegraf agent. When starting the container you have to map the Jolokia agent listening port along with any other port used by the Java application:
sudo docker run -d -p 8778:8778 ...
Assuming your Telegraf agent is configured to send data to InfluxDB, you should see in the database a table named jolokia, which contains the following fields keys:
name: jolokia fieldKey fieldType -------- --------- class_count_LoadedClassCount float class_count_TotalLoadedClassCount float class_count_UnloadedClassCount float heap_memory_usage_committed float heap_memory_usage_init float heap_memory_usage_max float heap_memory_usage_used float thread_count_DaemonThreadCount float thread_count_PeakThreadCount float thread_count_ThreadCount float thread_count_TotalStartedThreadCount float
Once understood how to set up the proper configurations, the process to implement the monitoring of Java applications inside Docker containers using Telegraf is pretty straightforward.