Debug the Nebula Graph Processes Inside the Docker Container
Join the DZone community and get the full member experience.Join For Free
In the development or testing process, we often use the vesoft-inc/nebula-docker-compose repository to deploy Nebula Graph. But to compress the docker image of each Nebula Graph service as much as possible, all the tools commonly used for development are not installed in the Docker images, not even the editor Vim.
This makes it difficult to locate problems inside the containers because it was cumbersome to install essential toolkit every time to get things done. In fact, there is another way to debug a process inside a container, without breaking the content structure of the container or installing any toolkit in it.
Actually, this technique, the Sidecar mode, is commonly used in the K8S environment. The principle is quite simple: Start a container that shares the same namespaces of the PID/network with the container we are debugging. In this way, we can view the processes and network namespaces in the original container through the container we use to debug, which has everything we need installed in it.
Here’s how to do it.
Start by deploying a Nebula Graph cluster locally with Docker-Compose, as we mentioned above. For a detailed tutorial, see the README file in the repository. After the deployment, we start all the services and check their status as follows.
Next, we demonstrate scenario by scenario, from the process namespaces to the network namespaces. First of all, we need to have a handy image for debugging. There is no need to build one by ourselves since this is only for demonstration. Let’s find a well-packed image from Docker Hub. If later we find this image not good enough, we can maintain a nebula-debug image and install all the debugging tools we want then. Here, we use a community solution from nicolaka/netshoot. Let’s pull the image to the local host.
Let's see what is going to happen if we run the image.
As shown above, this container does not have any Nebula Graph process. Let’s add a few parameters to it and see what happens.
This time it’s a bit different. We can see the metad0 process, which has the PID 1. Now we can easily do our work since we can see the process, for example, attaching it in the GDB. I don’t have an image with the Nebula Graph binary file handy, so I’ll leave the exploration to you.
We see that the PID namespace is shared already by setting
--pid container:<container_name|id>. Next, considering we might need to capture a package sometimes, let’s check if we can also see the network status. Run the following command.
There’s nothing. It’s a little bit different from what we expected, because we already have the metad0 process, and there should not be nothing here, not even one connection. To see the network namespaces within the original container, we need to set a few more options and run the following command to restart the debug container.
This time the output is not the same with the preceding one. With the
--network container:nebula-docker-compose_metad0_1 option, we can check the connections in the metad0 container, capture packages, and debug.
The key to being able to debug a container environment without installing extra tools in it is to run another container and have it share the PID/network namespaces with the original container. Some people in the community have even developed tools based on this method and made them easier to use. For more information, see Docker-debug.
Published at DZone with permission of Jamie Liu. See the original article here.
Opinions expressed by DZone contributors are their own.