Since its debut in 2013, Docker has exploded in popularity, and the company has completely revolutionized the way we manage applications. Articles no longer discuss the reasons why you should be adopting containers, but speak more about dealing with the challenges involved with widespread container development. Containers add speed and boost performance within the development process, but bring additional complexity in the form logging, orchestration, security, and visibility. However, dealing with scaling issues, their ephemeral nature, and the sheer volume of data involved with container clusters are the challenges that container monitoring solutions like Prometheus and Sysdig can help to address.

This guide is designed to help you compare two of the major solutions available for collecting the metrics you need. We’ll examine Prometheus and Sysdig—both highly popular container monitoring solutions—alongside the visualization tools (Grafana and Sysdig Monitor respectively) that complement them both to give you full visibility inside every container.

I’ve created evaluation criteria which assesses the two container monitoring solutions by the ease of deployment; the intricacy of the metrics the tool provides; the aggregation level; and the alerting capability each solution offers. For the visualization tools, I’ve evaluated the installation; the level of integration with its host monitoring solution; how easy it is to customize to your own specifications; and the general layout and feel.

Prometheus and Grafana

Developed by Soundcloud, Prometheus is an open source self-hosted monitoring solution which offers a wide set of tools for multi-dimensional metrics including aggregation, alerting, storage, and visualization. The query language is flexible and easy to grasp, and you can use the same language for graphing and alerting, making the whole container monitoring task much simpler.

Furthermore, you can also make Prometheus highly available by running identical Prometheus servers in different data centers/zones. That way, if a zone goes down your monitoring system in remaining zones can still be accessed.

Grafana

To visualize your data, Prometheus recommends Grafana. The two complement each other well as Grafana—just like Prometheus— uses various data source types. It takes a little time to build a fully-fledged dashboard using Grafana, but there’s a ton of documentation available to help you through the process. Plus, once built, Grafana dashboards can also be easily expanded to cover more than one Docker host. To monitor more hosts, simply deploy a node-exporter and a cAdvisor container on each host and tell the Prometheus server to scrape those as well.

Prometheus Pros

• It’s a powerful and easy to use monitoring tool;
• It deploys your whole stack using containers;
• The solution is built for distributed systems and infrastructure;
• It provides scalable data collection which is not dependant on distributed storage;
• It scales well due to a pull-based approach (fetches all metrics of a target over HTTP);
• It has built-in support for Azure, Consul, Docker, EC2, and Kubernetes with flexible service discovery;
• It supports and active and responsive community;
• There is no cost involved.

The Prometheus ecosystem is huge, which means you can find client libraries for many programming languages including Java, Go, Python, .NET, PHP, Ruby, etc.

Prometheus Cons

• It's not built as a dashboarding solution; requires Grafana to visualize metrics;
• It requires a lot of trial and error to set up both the way you want them;
• It doesn’t offer durable long-term storage or anomaly detection;
• It doesn’t provide automatic horizontal scaling or user management which large-scale enterprises may require.

Prometheus Metrics Review

Scorecard (Marks out of 5)

• Ease of deployment: ✔✔✔
• Metrics detail: ✔✔✔✔✔
• Aggregation level: ✔✔✔✔✔
• Alerting capability: ✔✔✔✔

Grafana Metrics Review

Scorecard (Marks out of 5)

• Ease of install: ✔✔✔
• Level of integration (with Prometheus): ✔✔✔✔
• Ease of customization: ✔✔✔✔
• Layout and feel: ✔✔✔✔✔

Sysdig and Sysdig Monitor

Another self-hosted service, Sysdig also provides all the necessary container orchestration tools to simplify Docker monitoring, including metrics aggregation and more.

Sysdig actually comes in two parts: the open source version, which requires you to install kernel headers on your host OS, and the cloud/on-premises aspect, which uses the open version to stream the metrics it collects from Sysdig’s own servers. As the solution hooks into the host’s kernel, it doesn't rely on getting metrics from the Docker Daemon. In addition, just as when running the Docker Stats command and using cAdvisor, Sysdig’s open source version allows you to get a real-time view of your containers.

Sysdig Monitor

Sysdig Monitor—formerly Sysdig Cloud—automatically discovers all the containers in your environment and displays them on your choice of several preconfigured dashboards. It captures application, container, host, and StatsD metrics all with a single collection point. The solution provides great visualization tools for real-time and historical data; in either a graphical or tabular representation of your deployment. As well as being able to drill into containers’ individual processes, the dashboards have great alerting functionality should incidents occur.

Sysdig Pros

• Native support for all Linux tech, including Docker, Kubernetes, Mesos, etc.;
• Collect all types of data including Docker and Kubernetes event logs, as well as metadata from container orchestration tools;
• Filters for most common services, including Apache Tomcat, NGINX, MongoDB, PHP-FPM, and PostgreSQL, for insight into database performance;
• Can dive into individual containers to view what resources individual processes are using;
• Troubleshoot by pod, cluster, namespace, etc. by hooking into your orchestration tools;
• Record and replay system activity;
• Alerting function allows you to target different email addresses for different alerts;
• Team functionality which lets admins control team access to the dashboards, data, and alerts.

Sysdig Cons

• The need to install kernel headers on the host OS is a pain;
• It takes a bit of time to get used to the UI
  
• Paid solution starting at $20 with pricing variations according to requirements which can make it a costly solution in comparison to Prometheus.

Sysdig Metrics Review

Scorecard (Marks out of 5)

• Ease of deployment: ✔✔✔
• Metrics detail: ✔✔✔✔✔
• Aggregation level: ✔✔✔✔✔
• Alerting capability: ✔✔✔✔

Sysdig Monitor Metrics Review

Scorecard (Marks out of 5)

• Ease of install: ✔✔✔✔
• Level of integration (with Sysdig): ✔✔✔✔✔
• Ease of customization: ✔✔✔✔
• Layout and feel: ✔✔✔

Overview

Fundamentally, choosing the right container monitoring solution will come down to a number of things. Top of which are going to be your requirements and, probably most importantly, your budget. Of the two monitoring tools, Sysdig (along with Sysdig Monitor) is far easier to setup and get working straight away for you and your team. In comparison, Prometheus and Grafana will require a little more time and patience to get configured—though the payback here is that the solution is then tailored to fit your exact demands.

In addition, Sysdig can become quite costly per year as a hosted solution, while Prometheus is open source and free. Both tools are backed by active communities which can be tapped into for help and support, though Prometheus’ is the larger and more established of the two. In short, the right tool for you will end up being the cost and your preference overall, as both will scale and likely achieve what you need.