Running Jenkins Build Slaves in A Docker Container
Running Jenkins Build Slaves in A Docker Container
Learn how to create and run Jenkins slave build machines from within a Docker container and trigger them automatically with the remote API.
Join the DZone community and get the full member experience.Join For Free
We all know that every organization contains many build machines when there are larger and heavier projects that build all the time. These build machines are nothing special, just a physical machine with a communication created between this and the Jenkins server. Whenever a build needs to be done, the job from the Jenkins server is triggered and the Jenkins server will run the job on a build machine identified by a label. These build machines that we create and connect to the Jenkins server are called slaves. Each of the slaves will have a label to be identified by the Jenkins server for running the job.
With the arrival of containers, there is no need for separate build physical machines. We can configure Jenkins to run our jobs in one of the containers that gets created; the job is run inside the container and destroyed once complete. In this article, we will see how we can run a Docker container as our slave and run a job inside the slave.
In this article, we will see how we can configure Jenkins to trigger a Docker container whenever we want to run a job. We will have Jenkins running on a different machine and have Docker slave containers created on a different machine.
You may also enjoy: Get Started With Jenkins 2.0 With Docker
Configure Docker to Enable the Remote API
The remote API allows us to execute the Docker commands using the API. Add the below contents to the /etc/docker/daemon.json file
Restart the Docker daemon and Docker server using the command:
sudo systemctl daemon-reloadsudo systemctl restart docker.service
Confirm the remote API by making a call:
Use the curl command and access “http://localhost:2375/version” and if we see the output is working .
For the purpose of the article, I have taken an AWS EC2 instance with the IP address “18.104.22.168.” installed Docker, and configured the Docker API.
On the Jenkins side, install the Docker plugin from the “Manage Jenkins -> Manage plugins.” Once we install the plugin, we will see an entry added to the “Manage Jenkins -> Configure System” called “cloud.”
Configure the “Docker Cloud Details” as you see below:
In the configuration above, the Docker Host URI is the important one. This is the same machine where I want to run my Docker slave containers. In my case, I have taken an AWS EC2 instance with the IP address “22.214.171.124.” The Docker Host URI is “tcp://126.96.36.199:2375.” This is the same host configuration that we did in the first step. Enable the “Enabled” checkbox. You can keep the other fields as they are.
One important thing is to make sure the remote API is working by clicking the “Test Connection” button. Once it returns the version and API version values as shown in the above image, it's working fine.
Once the Docker cloud is configured, we need to configure the Docker template, which is below the Docker cloud in the same page.
There are many configurations that we need to do in this template. The label is the first thing that we need to configure. First, give a name to the label field; this is the same name that we will use in a job to run the project. I have given the label “slave” and when I configure the Jenkins job, I will set the label name in the job to“slave” so that that job will run in this Docker container. You may choose any name you see fit.
The next and most important element is the Docker image. There are certain conditions when running a slave container for Jenkins. First, we need to build a Docker image by taking the “jenkins/ssh-slave” as the base image.
Then we can install the necessary packages in it. This image will then need to be saved to the Docker Hub. Below is the Dockerfile for this article
I built the image and saved it with the name “docker.io/jagadesh1982/jenkins-slave-test:latest.” I then used the
docker push command to push to the Docker Hub.
Now, once we define the image details in the “Docker template” in Jenkins, Jenkins will download the image to that machine and run the container using this image. One important thing is that we have a “Registry Authentication” button which will let us configure our Docker Hub credentials as Jenkins credentials. Jenkins will use these credentials to log in to the Docker Hub and download the image to the machine. We need to configure these credentials with your Docker Hub details.
The instance capacity is set to 5 and the home location is set to “/home/jenkins.” The next important thing is the connection method. In this case, I choose the “Connect on ssh” which will show the details below. In the ssh key, I have chosen the “Inject ssh key” type, which will inject a dedicated ssh key into the container and the sshd is configured accordingly. The user will be Jenkins since we built the image from the Jenkins/ssh-slave. We can choose our own pull strategy.
Create a sample Freestyle Job and set the label for the job by clicking the option that says, “Restrict where the project can be run." In this field, enter the label “slave” that we created for the Docker cloud earlier.
Now if we run the job, we can see the image file is download and started in the EC2 instance as below:
We can see that the image file is downloaded and the container is started just three seconds before. Happy learning!
Opinions expressed by DZone contributors are their own.