Get Alerts When Docker Containers Run Into Issues

I'm running Docker containers for all my side projects, usually one single container, sometimes multiple containers started by docker-compose[1].

If containers get issues, I want to get alerts.

What a typical monitoring requirement! But if you do some research, you will find the information is just overwhelming. cAdvisor, Prometheus, InfluxDB, etc. Excuse me? Can't we have a simple solution for this simple requirement? Here is my answer. Try it and discuss with me.

This solution works for docker-compose deployments. Please check Prometheus[2] for more complex Docker environments.

Basic Idea

First, I admit: Prometheus can do the job. Yes, it's capable, but ...

Hint: If you want to use Prometheus in a relatively easy way, try dockprom[3].

Frankly speaking, learning Prometheus took much longer than what I had expected. And it's a bit over-killing for my scenario. A Prometheus solution itself will start multiple containers. But my envs only have very few containers to monitor.

Ironically "docker ps -a" can easily answer most of my questions.

root@denny-blog:~# docker ps -a
CONTAINER ID    IMAGE             COMMAND                  CREATED         STATUS                     PORTS                   NAMES
2936e1d7ec34    denny_wordpress   "/bin/sh -c /docke..."   3 weeks ago     Up 13 days (healthy)       0.0.0.0:443->443/tcp    blog-wordpress
87665b7a76df    mysql:5.7         "docker-entrypoint..."   3 weeks ago     Up 3 weeks                 3306/tcp                blog-mysql
4aa821fa9d37    denny/ss:v2       "ssserver -c /etc/..."   3 weeks ago     Up 3 weeks (healthy)       0.0.0.0:6187->6187/tcp  shadowsock
ea96f4dbefaa    ubuntu:14.04      "/bin/bash"              14 seconds ago  Exited (0) 6 seconds ago                           nginx-test

1. To get stopped containers: just check the "STATUS" of each container.
   
2. To get unhealthy containers: check the status for "unhealthy" output.
   
   Hint: To enable Docker healthcheck, you need to update your Dockerfiles properly.
   
3. Send alerts whenever any of above occurrences have happened. Should be easy, right?
   

So how about we start a container, which keeps polling status of all containers, then send out alerts if necessary?

You may think: we won't be able to run "docker ps" in containers. It only runs in Docker host.

Yes, you're right. But there is one way out. You can query the status via the Docker Unix socket file. Try it below locally; you will understand.

docker run -v /var/run/docker.sock:/var/run/docker.sock \
    -t -d --privileged --name test-socket \
    --entrypoint=/bin/sh byrnedo/alpine-curl

docker exec -it test-socket \
    curl -XGET --unix-socket /var/run/docker.sock \
    http://localhost/containers/json

How to Use

Check the code in GitHub.

Try it if you find it's useful, and discuss it with me.