Ever need to debug your process in containers? Use strace, lsof, pstree, or anything you name it. But after login, you get a surprise: The tools are not installed!
So what will you do? Typically we have 3 different methods. Check it out, and discuss with me.
(Hint: You don’t have to install the tools at all.)
Start an Nginx container for an explanation. We will run tools against the Nginx process afterwards.
# pull docker image docker pull nginx:alpine # start container docker run -t -d --privileged \ -h nginxtest --name nginx-test \ -p 8080:80 nginx:alpine
Check the Nginx process:
# verify nginx httpd service curl http://localhost:8080 # check process pid docker exec nginx-test ps -ef | grep nginx
Method 1: Debug From Inside the Container
Let’s say you want to “strace -p $nginx_pid.” But strace is not available in the nginx:alpine image.
Login and install. Yes, it will work. But just old school.
docker exec -u root -it nginx-test sh # Install strace apk --update add strace # strace nginx process strace -p 1 ## strace: Process 1 attached ## rt_sigsuspend(, 8
Why is it old school? It will populate the env. Especially when the containers are in production mode.
The more packages you have installed, the more issues you will get.
Method 2: Debug From Docker Host
Linux containers share the same Linux kernel.
We can find the process id from docker host, then debug the process.
# get nginx process pid root@denny:/tmp# ps -ef | grep nginx root 27871 27834 0 22:00 pts/2 00:00:00 nginx: master process nginx -g daemon off; systemd+ 27931 27871 0 22:00 pts/2 00:00:00 nginx: worker process root 31324 27756 0 23:26 pts/1 00:00:00 grep --color=auto nginx # Install tools in docker host apt-get install -y strace # Run tools from docker host root@denny:/tmp# strace -p 27871 strace: Process 27871 attached rt_sigsuspend(, 8
Method 3: Debug From an Ephemeral Container
Build a temporary image with tools installed.
# Dockerfile cat > Dockerfile <<EOF FROM alpine RUN apk update && apk add strace CMD ["strace", "-p", "1"] EOF # Build image docker build -t strace .
Start a temporary container(nginx-test). Then debug the Nginx process by strace.
export test_conainter="nginx-test" docker run -t --name strace-test \ --pid=container:$test_conainter \ --net=container:$test_conainter \ --cap-add sys_admin \ --cap-add sys_ptrace \ strace
Don’t forget to destroy the container when you have finished your debugging.
Apparently, I like Method 3 the best. In this article, we use the strace tool. Surely we can support more tools like this.
Not so difficult as you thought. Right, my friend?
So why don’t you give it a try now? Or share this post with your colleagues?