Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

All-in-One Docker with Grafana, InfluxDB, and Cloudwatch-to-Graphite for AWS/Beanstalk Monitoring

DZone's Guide to

All-in-One Docker with Grafana, InfluxDB, and Cloudwatch-to-Graphite for AWS/Beanstalk Monitoring

Learn how to use a Docker container that bundles Grafana dashboards, and InfluxDB to make effective use of AWS CloudWatch to fetch selected metrics.

· DevOps Zone
Free Resource

Download the blueprint that can take a company of any maturity level all the way up to enterprise-scale continuous delivery using a combination of Automic Release Automation, Automic’s 20+ years of business automation experience, and the proven tools and practices the company is already leveraging.

I have derived the Docker container docker-grafana-influxdb-cloudwatch that bundles Grafana dashboards, InfluxDB for metrics storage, and runs cloudwatch-to-graphite as a cron job to fetch selected metrics from AWS CloudWatch and feed them into the InfluxDB using its Graphite input plugin. It is configured so that you can run it in AWS Elastic Beanstalk (the main problem being that only a single port can be exposed – I therefore use Nginx to expose the InfluxDB API needed by Grafana at :80/db/).

Highlights

  • It’s derived from kamon-io/docker-grafana-influxdb but based on phusion/baseimage rather than raw Ubuntu as this supports cron and presumably solves some deficiencies wrt. Docker
  • InfluxDB API is exposed also at port 80 so that it can be easily run in Elastic BeanStalk
  • It enables the Graphite InfluxDB plugin
  • It runs cloudwatch-to-graphite’s leadbutt to fetch metrics from CloudWatch
  • Grafana configuration fixed not to hardcode InfluxDB as locahost

Why?

I’ve created this to be able to run a monitoring dashboard on Elastic Beanstalk to get overview of our AWS infrastructure.

  • CloudWatch only keeps metrics for the past 2 week, doesn’t provide customizable dashboards, and cannot be exposed without the need for authentication
  • Beanstalk supports running docker containers – I just need to upload a .zip with the Dockerfile and files (it also supports multiple container setups but then I’d need to publish the images into a repository; the same holds for Amazon EC2 Container Service

Limitations

  • I haven’t managed yet to make the InfluxDB storage persistent (preferably stored on an EBS volume) though I’ve tried (ideas welcome!); a re-deployment results in loss of data
  • You want to keep the number of metrics collected minimal as you have only 1M API calls free and it can get expensive for a large infrastructure
  • This is likely not suitable for high load and large volumes of data

Get it!

https://github.com/jakubholynet/docker-grafana-influxdb-cloudwatch

Download the ‘Practical Blueprint to Continuous Delivery’ to learn how Automic Release Automation can help you begin or continue your company’s digital transformation.

Topics:
java ,cloud ,devops ,docker ,ops ,amazon web services

Published at DZone with permission of Jakub Holý, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}