Sending Custom Application Stats From EC2 Server to CloudWatch for Monitoring

In AWS, one of the common challenges that we all of us face is how to send custom application stats to CloudWatch for monitoring like we send memory, CPU, and I/O stats for automatic monitoring. I did this in one of my recent projects, and I want to share that with all of you through this article. 

This approach works fine for:

• Any custom application, that runs as a service.

• Third-party applications, like Apache Tomcat, which runs as a service on some EC2 Linux instance.

For illustration purposes, I will be taking an example of Apache Tomcat version 9, which is commonly used in many applications by many of you, so that you all can correlate with this and use for other purposes. Do leave me a comment if you need additional help in this, I will try to respond to your queries. 

To achieve this, we will do it in three steps:

1. Creating a custom bash script, which will check the status of the application.

2. Setting up a CRON job, which will send the collected stats to CloudWatch server.

3. Creating Alarms on CloudWatch server so that some action can be taken if some service goes down.

Step 1 

Apache Tomcat 9 is running on my EC2 Linux instance. Just create a bash script in the /opt/ folder with the name: publish-tomcat-cloudwatch.sh You can use vim or vi or nano or any other file editor to create below file, based on your interest and availability.

#!/bin/bash

#set -x
INSTANCE_ID=$(/opt/aws/bin/ec2-metadata -i | cut -d " " -f2)

checkTomcatStatus(){
        counter=0
        ps x | grep /opt/apache-tomcat-9.0.12/ | grep -v grep |
        while read -r LINE
        do
                read PROCESS_ID <<< $LINE
                counter=$((counter+1))
                echo $counter
        done
}

i=$(checkTomcatStatus)
#echo $i

if [ "$i" == "" ]
then
        aws --region eu-west-1 cloudwatch put-metric-data --metric-name tomcat --value 0 --namespace tomcat --dimensions InstanceId=$INSTANCE_ID
else
        aws --region eu-west-1 cloudwatch put-metric-data --metric-name tomcat --value 1 --namespace tomcat --dimensions InstanceId=$INSTANCE_ID
fi

Note: AWS CLI must be configured correctly on your EC2 instance for this script to work correctly, as this script uses that to post the service details on CloudWatch server. Secondly, your instance must have the proper permissions to publish stats to CloudWatch server. Please refer AWS documentation for more details on this.

This script checks the status of Apache Tomcat 9 to see whether it is running or not, and publishes "1" if it is running, "0" otherwise, to the CloudWatch server. Give full permissions to this script file so that it can run without any issue using the following command.

# chmod 777 /opt/publish-tomcat-cloudwatch.sh

Step 2

Now, as we have created the script which will send the data to CloudWatch server, it's the time to automate a process by which this script should automatically send the collected stats of Apache Tomcat 9 to the CloudWatch server. To achieve the same, let's create a cron job.

In Linux, we will use crontab to schedule the job after frequent intervals of five minutes, for example.  We can do it in two ways:

echo "*/5 * * * * /bin/sh /opt/publish-tomcat-cloudwatch.sh" > /var/spool/cron/root

or using the following command:

# crontab -e
*/5 * * * * /bin/sh /opt/publish-tomcat-cloudwatch.sh

"Save the file using Esc+:wq"

Now you can see the cron job is running and publishing the stats to the CloudWatch Server.

Step 3

If you login to your AWS Console, and navigate to CloudWatch service, you can see that the data is now available for you. If the Tomcat is up, you will see "1" and if it is down, you can see the matrix floating down to "0." You can create a custom alarm if the Tomcat goes down. 

Please refer to the AWS documentation for more information. 

Congratulations, you did it!

Do share your thoughts if you have any queries or suggestions. 

Thanks!