Over a million developers have joined DZone.

Monitoring Docker Container Health

It's important to monitor your infrastructure, and Docker containers are no exception. But how? Find out here.

· DevOps Zone

The DevOps zone is brought to you in partnership with Sonatype Nexus. The Nexus suite helps scale your DevOps delivery with continuous component intelligence integrated into development tools, including Eclipse, IntelliJ, Jenkins, Bamboo, SonarQube and more. Schedule a demo today

In the latest Docker release, version 1.12, the new health check instruction for Dockerfiles reflects the changing nature of application development. The combination of dynamic infrastructures spanning multiple data centers, complex service dependencies, and stringent uptime requirements is dramatically affecting how individual applications are designed and deployed. At the same time, with the increased popularity of microservices, dev and ops teams are working with smaller, more frequently deployed services.

Docker’s health check instruction supplies important additional information for running containers in these increasingly complex environments. While the “docker ps” command makes it easy to determine if a container is running, a health check lets you specify a command in a Dockerfile for a container-specific way to determine readiness. We’ll go through a simple example of using the instruction with an existing Node.js application that we had created for this post on The Modern Developer Workstation on MacOS With Docker.

docker health check: doctor

Your Container Is Running. But Is It Healthy?

Apart from processing exit codes, Docker, by design, doesn’t know much about the internal workings of containerized applications. When “docker run” is invoked from the command-line interface (CLI), it often starts a single process specified using the CMDinstruction in a Dockerfile. That determines the result of the “STATUS” column in the docker CLI with the ps command. The -a flag lists all containers, running or not, as shown here:

docker health check screenshot

Output of “docker ps -a”

This output indicates that some containers are actively running while others have exited, either successfully or with an error (“Exited (1)”). Unfortunately, even applications that show a status of “Up 16 minutes” may serve 500 errors or could be stuck in an infinite loop. Adding HEALTHCHECK to the container Dockerfile helps address this issue.

Writing Your First Health Check

Health checks can be any single command. They run inside the container and if the command exit code is 0, the container is reported as healthy, and if the output is 1, the container is marked as unhealthy.

By default, as of the latest Docker 1.21.1 RC release, the health check command runs every 30 seconds, retries three times before failing, and times out after 30 seconds. These values are all configurable and should ideally be related to a service’s SLA requirements and tuned during game-day simulations.

Unless specifically disabled or overridden in the Dockerfile, it is also inherited from base images. This lets developers define a standard health check for similar applications.

For example, a common verification for web-facing application containers checks the HTTP status code from an app running in a container. Using curl’s --fail option, this command can return a status code 1 if an HTTP error is returned from the service (running on port 3000 in this example):

HEALTHCHECK CMD curl --fail http://localhost:3000/ || exit 1

After the container image is rebuilt and run, the health state is returned in parentheses in the output of docker ps:

docker health check screenshot

If a code change causes the application to send an error status code (such as 500), the container health status changes to unhealthy after three failures (the default) because the check command returns “1”:

docker health check screenshot

Health checks take time to complete. When a health check is executing for the first time, the status is shown as “starting:

docker health check screenshot

Any health changes trigger a Docker event (heath_status) so you can react to changes without resorting to polling the Docker engine. Although this instruction is still new, it promises to help developers build more resilient software in a variety of scenarios: deployment and software load balancing, for example.

Finally, for debugging health checks, the Docker inspect command lets you view the output of commands that succeed or fail (here in JSON format):

docker inspect --format='{{json .State.Health}}' your-container-name

Health Check Meets Complex Deploys and Orchestration

As of early July 2016, the new orchestration features in Docker Swarm mode services are utilizing a health check to manage zero-downtime deployments. Only when the status changes to “healthy” will it start routing traffic to container instances. This pattern is also used by other popular orchestration frameworks. For development and operations teams, health checks can help avoid an unfortunate failure scenario — code deployed to production that runs but always returns a mysterious internal server error due to a configuration error.

Regardless of whether your team is using bleeding-edge orchestration or not, Docker’s configurable health check instruction allows teams to define their own idea of what makes their application running in a container “ready.” As discussed in my article, Writing Better HTTP Health Checks for Complex Infrastructure, the time and effort put into this pattern is ultimately about team health. Service health checks can make increasingly large and complex systems easier to manage, deploy, and troubleshoot.

The DevOps zone is brought to you in partnership with Sonatype Nexus. Use the Nexus Suite to automate your software supply chain and ensure you're using the highest quality open source components at every step of the development lifecycle. Get Nexus today


Published at DZone with permission of Clay Smith, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}