Full Build Automation For Java Application Using Docker Containers

In this pipeline implementation, we will be using Dockers containers for building our Java application. We will run Jenkins inside a Docker container, start Maven containers from the Jenkins container to build our code, run test cases in another Maven container, generate the artifact (jar in this case), then build a Docker image inside the Jenkins container itself and push that to the Docker Hub at the end from Jenkins Container.

For this Pipeline, we will be using 2 Github repositories.

1. Jenkins-complete: This is the main repository. This repo contains configuration files for starting Jenkins Container.

2. Simple-java-maven-app: This is our sample Java application created using Maven. 

We need to understand both repos before building this automation. 

Understanding Jenkins-complete

This is the core repository as this will contain all the necessary files that build our Jenkins image. Jenkins officially provides a Docker image by which we can start the container. Once the container is started, we need to perform many activities, such as installing plugins, creating a user, and more. 

Once these are created, we need them to create credentials for Github to check our sample Java application, and a Docker credential for pushing the created Docker image to Dockerhub. Finally, we have to create the pipeline job in Jenkins for building our application.

This is a long process; our goal is to completely automate all these things. This repo contains files and configuration details which will be used while creating the image. Once the image is created and run, we have:

1. User admin/admin created
2. Plugins installed
3. Credentials for Github and Docker are created
4. Pipeline job with name sample-maven-job is created.

If we check out the source code and do a tree, we can see the below structure,

jagadishmanchala@Jagadish-Local:/Volumes/Work$ tree jenkins-complete/
jenkins-complete/
├── Dockerfile
├── README.md
├── credentials.xml
├── default-user.groovy
├── executors.groovy
├── install-plugins.sh
├── sample-maven-job_config.xml
├── create-credential.groovy
└── trigger-job.sh

Let's see what each file talks about

default-user.groovy - this is the file that creates the default user admin/admin.

executors.groovy - this is the Groovy script that sets the executors in the Jenkins server with value 5. A Jenkins executor can be treated as a single process which allow a Jenkins job to run on a respective slave/agent machine.

create-credential.groovy - Groovy script for creating credentials in the Jenkins global store. This file can be used to create any credential in the Jenkins global store. This file is used to create Docker hub credentials. We need to change the username and secret entries in the file by adding our Docker hub username and password. This file will be copied to the image and ran when the server starts up

credentials.xml - this is the XML file which will contain our credentials. This file contains credentials for both Github and Docker. The credential looks like this:

<com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl>
          <scope>GLOBAL</scope>
          <id>github</id>
          <description>github</description>
          <username>jagadish***</username>
   <password>{AQAAABAAAAAQoj3DDFSH1******</password>  
   </com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl>

If you see the above snippet, we have the id as “github,” username and encrypted password. The id is very important as we will be using this in the pipeline job that we create.

How Can I Get the Encrypted Values for My Password?

In order to get encrypted content for your password, go to a running Jenkins server -> Manage Jenkins -> Script console. In the text field, it provides the option enter the code,

import hudson.util.Secret
def secret = Secret.fromString("password")
println(secret.getEncryptedValue())

In the place of a password, enter your password which, when run, gives you the encrypted password. You can paste the content in the credentials.xml file.

The same thing is used to generate the DockerHub password, too. 

sample-maven-job_config.xml - this is the XML file which contains our pipeline job details. This will be used by the Jenkins to create a job by the name “sample-maven-job” in the Jenkins console. This job will be configured with the details defined in this XML file.

The configuration is simple, Jenkins will read this file to create a job “sample-maven-job” pipeline job, sets the SCM pointing to the Github location. This will also be configured with the credential set to “github” id. This looks something like this,

Once the scm is set, it also set the job with a Token for triggering the job remotely. For this, we have to enable the “Trigger builds remotely” option and provide a token over there. This is available under the “Build Triggers” section in the pipeline job. I have given the token as “MY-TOKEN” which will be used in our shell script to trigger the job.

trigger-job.sh - This is a simple shell script which contains a curl command for running the job.

Though we create the entire Jenkins Server inside a container, and create a job, we need a way to trigger that job in order to make that whole build automated. I preferred a way of:

1. Creating the Jenkins Docker container with all necessary things like job creation, credentials, users, etc.
2. Trigger the job once the container is up and running.

I have written this simple shell script to trigger the job once the container is up and running. The shell script is a simple curl command sending a post request to the Jenkins server. The content looks something like this.

Install-plugins.sh - This is the script that we will use to install the necessary plugins. We will copy this script to the Jenkins images passing the plugin names as arguments. Once the container is started, the script is ran taking the plugins as arguments and are installed.

Dockerfile - The most important file in this automation. We will use the Docker file to build the whole Jenkins server with all configurations. Understanding this file is very important if you want to write your own build automations.

FROM jenkins/jenkins:lts
ARG HOST_DOCKER_GROUP_ID
# Installing the plugins we need using the in-built install-plugins.sh script
RUN install-plugins.sh pipeline-graph-analysis:1.9 \
    cloudbees-folder:6.7 \
    docker-commons:1.14 \
    jdk-tool:1.2 \
    script-security:1.56 \
    pipeline-rest-api:2.10 \
    command-launcher:1.3 \
    docker-workflow:1.18 \
    docker-plugin:1.1.6

# Setting up environment variables for Jenkins admin user
ENV JENKINS_USER admin
ENV JENKINS_PASS admin

# Skip the initial setup wizard
ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false

# Start-up scripts to set number of executors and creating the admin user
COPY executors.groovy /usr/share/jenkins/ref/init.groovy.d/
COPY default-user.groovy /usr/share/jenkins/ref/init.groovy.d/
COPY create-credential.groovy /usr/share/jenkins/ref/init.groovy.d/

# Name the jobs  
ARG job_name_1="sample-maven-job"
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/latest/  
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/builds/1/
COPY ${job_name_1}_config.xml /usr/share/jenkins/ref/jobs/${job_name_1}/config.xml
COPY credentials.xml /usr/share/jenkins/ref/
COPY trigger-job.sh /usr/share/jenkins/ref/

# Add the custom configs to the container  
#COPY ${job_name_1}_config.xml "$JENKINS_HOME"/jobs/${job_name_1}/config.xml  
USER root
#RUN chown -R jenkins:jenkins "$JENKINS_HOME"/
RUN chmod -R 777 /usr/share/jenkins/ref/trigger-job.sh

# Create 'docker' group with provided group ID
# and add 'jenkins' user to it
RUN groupadd docker -g ${HOST_DOCKER_GROUP_ID} && \
    usermod -a -G docker jenkins

RUN apt-get update && apt-get install -y tree nano curl sudo
RUN curl https://get.docker.com/builds/Linux/x86_64/docker-latest.tgz | tar xvz -C /tmp/ && mv /tmp/docker/docker /usr/bin/docker
RUN curl -L "https://github.com/docker/compose/releases/download/1.23.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
RUN chmod 755 /usr/local/bin/docker-compose
RUN usermod -a -G sudo jenkins
RUN echo "jenkins ALL=(ALL:ALL) NOPASSWD:ALL" >> /etc/sudoers
RUN newgrp docker
USER jenkins
#ENTRYPOINT ["/bin/sh -c /var/jenkins_home/trigger-job.sh"]

FROM jenkins/jenkins:lts - We will be using the Jenkins image provided officially. 

ARG HOST_DOCKER_GROUP_ID  - One important thing to keep in mind is that, though we create Docker containers from the Jenkins Docker container, we are not actually creating containers inside Jenkins. Rather, we are creating them on the host machine itself. This means we tell the Docker tool installed inside the Jenkins Docker container to delegate the request of creating the Maven container to the host machine. In order for the delegation to happen, we need to have the same groups configured on the Jenkins Docker container and on the host machine.

To allow access for a non-privileged user like Jenkins, we need to add the Jenkins user to the Docker group. In order to make things works, we must ensure that the Docker group inside the container has the same GID as the group on the host machine. The group id can be obtained using the command  getent group Docker .

Now the HOST_DOCKER_GROUP_ID is set as build argument which means we need to send the group ID for Docker on the host machine to the image file while building this. We will be sending this value as an argument which builds that.

# Installing the plugins we need using the in-built install-plugins.sh script
RUN install-plugins.sh pipeline-graph-analysis:1.9 \
    cloudbees-folder:6.7 \
    docker-commons:1.14 \

The next instruction is to run the “install-plugins.sh” script, passing the plugins to be installed as arguments. The script is provided by default or we can copy from our host machine.

Setting up Environment Variables for Jenkins Admin User

ENV JENKINS_USER admin
ENV JENKINS_PASS admin

We set the JENKINS_USER and JENKINS_PASS environment variables. These variables are passed to the default-user.groovy script for creating user admin with password admin.

# Skip the initial setup wizard
ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false

This lets Jenkins be installed in silent mode

# Start-up scripts to set number of executors and creating the admin user
COPY executors.groovy /usr/share/jenkins/ref/init.groovy.d/
COPY default-user.groovy /usr/share/jenkins/ref/init.groovy.d/
COPY create-credential.groovy /usr/share/jenkins/ref/init.groovy.d/

The above scripts as we discussed will set the executors to 5 and create a default user admin/admin.

One important thing to remember here is, if we check the Jenkins official Docker image we will see a VOLUME set to /var/jenkins_home. This means this is the home directory for our Jenkins server similar to /var/lib/jenkins when we install on physical machine.

But once a volume is attached, the only root user has the capability to edit and add files over there. In order to let non-privileged user “jenkins” copy content to this location, copy everything to the location /usr/share/Jenkins/ref/. Once the container is started, the Jenkins will take care of copying the content from this location to the /var/jenkins_home as Jenkins users.

Similarly, scripts copied to /usr/share/jenkins/ref/init.groovy.d/ will be executed when the server gets started up.

# Name the jobs 
ARG job_name_1="sample-maven-job"
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/latest/ 
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/builds/1/
COPY ${job_name_1}_config.xml /usr/share/jenkins/ref/jobs/${job_name_1}/config.xml
COPY credentials.xml /usr/share/jenkins/ref/
COPY trigger-job.sh /usr/share/jenkins/ref/

In the above case, I am setting my job name as “sample-maven-job” and creating the directories and copying the files.

RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/latest/ 
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/builds/1/

These instructions are very important, as they create a jobs directory in the Jenkins home where we need to copy the job configuration file. The latest/ and builds/1 also need to be created in the job location for that specific job.

Once these are created, we are copying our “sample-maven-job_config.xml” file to the /var/share/jenkins/ref and asking Jenkins to copy the file to the /var/jenkins_home/jobs/ as sample-maven-job.

Finally, we are also copying the credentials.xml and trigger-job.sh files to the /usr/share/jenkins/ref. Once the container is started, all content available in this location will be moved to /var/jenkins_home as Jenkins user.

USER root
#RUN chown -R jenkins:jenkins "$JENKINS_HOME"/
RUN chmod -R 777 /usr/share/jenkins/ref/trigger-job.sh

# Create 'docker' group with provided group ID
# and add 'jenkins' user to it
RUN groupadd docker -g ${HOST_DOCKER_GROUP_ID} && \
    usermod -a -G docker jenkins

RUN apt-get update && apt-get install -y tree nano curl sudo
RUN curl https://get.docker.com/builds/Linux/x86_64/docker-latest.tgz | tar xvz -C /tmp/ && mv /tmp/docker/docker /usr/bin/docker
RUN curl -L "https://github.com/docker/compose/releases/download/1.23.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
RUN chmod 755 /usr/local/bin/docker-compose
RUN usermod -a -G sudo jenkins
RUN echo "jenkins ALL=(ALL:ALL) NOPASSWD:ALL" >> /etc/sudoers
RUN newgrp docker
USER jenkins

The next instructions are executed as the root user.  Under the root instructions, we are creating the Docker group with the same group ID as the host machine passed as an argument. Then we are modifying the Jenkins user by adding that to the Docker group. By this, we can create a container from the Jenkins user. This is very important as a Docker container can only be created by the root user. In order to create them with Jenkins, we need to add the Jenkins user to the Docker group.

In the next instructions, we are installing the docker-ce and docker-compose tools. We are also setting the permissions on the Docker-compose tool. Finally, we are also adding the Jenkins to the sudoers file to give certain permissions to the root user.

RUN newgrp docker

This instruction is very important. Whenever we modify the groups of a user, we need to log out and login to reflect the changes.  In order to bypass the logout and login we use this newgrp docker instruction to reflect the changes. Finally, we change back to the Jenkins user.

Build the Image

Once we are good with the Docker file, to create an image from this we need to run,

docker build --build-arg HOST_DOCKER_GROUP_ID="`getent group docker | cut -d':' -f3`" -t jenkins1 .

From the location where we have our Dockerfile, run the above Docker build instruction. In the above command, we are passing the build-arg with the value of the group ID for the Docker user. This value will be passed to the “HOST_DOCKER_GROUP_ID” which will be used to create the same group ID in the Jenkins Docker container. The image build will take some time since it needs to download and install the plugins for Jenkins servers.

Running the Image

Once the image is built, we need to run the container as

docker run -itd -v /var/run/docker.sock:/var/run/docker.sock  -v $(which docker):/usr/bin/docker -p 8880:8080 -p 50000:50000 jenkins1

Two important things in here are that volumes that we are mounting. We are mounting the Docker command line utility to the container, so that if another container needs to be created from the container, this can be used.

The most important one is the /var/run/Docker.sock mount. Docker.sock is a UNIX socket that Docker daemon is listening to. This is the main entrypoint for the Docker API. This can also be a TCP socket but by default, for security reasons, it is a UNIX socket.

Docker uses this socket to execute the Docker command by default. The reason why we mount this to a Docker container is to launch new containers from the container. This can also be used for auto service discovery and logging purpose. This increases attack surface so we need to be very careful mounting this.

Once the command is run, we will get the Jenkins container up and running. Use the URL “<ip address>:8880” to see the Jenkins console. Once the console is up, login using “admin/admin.” We will see our sample-maven-job created with SCM, Token and credentials But not ran yet

Running the Job

In order to run the job, all we have to do is to take the containerID and run the trigger-job.sh job as below,

docker exec <Jenkins Container ID> /bin/sh -C /var/jenkins_home/trigger-job.sh

Once you run the command we can see the building the pipeline job get started.

Understanding the Simple Java Maven App

As we already said,this repo contains our Java application. The application is created using Maven artifacts. The repo contains a Dockerfile, a Jenkinsfile and source code. The source code is quite similar to other Maven-based applications.

Jenkinsfile - The Jenkins file is the core file that will be run when the sample-maven-job is started. The pipeline job downloads the source code from the GitHub location using the GitHub credential.

The most important thing in the Jenkinsfile is the agent definition. We will be using “agent any” for building our Java code from any available agents. But we will define agents when we go to specific stages to run the stage.

stage("build"){
        agent {
                docker {
                    image 'maven:3-alpine'
                    args '-v /root/.m2:/root/.m2'
                  }
             }

       steps {
              sh 'mvn -B -DskipTests clean package'
              stash includes: 'target/*.jar', name: 'targetfiles'
             }

If you see the above stage, we are setting the agent as Docker with image file “maven:3-alpine.” So Jenkins will trigger a Docker run maven:3-alpine container and run the command defined in the steps as  mvn -B -DskipTests clean package .

Similarly, the test cases are also run in the same way. It triggers a docker run   with the Maven image and then runs mvn test on the source code.

environment {
    registry = "docker.io/<user name>/<image Name>"
    registryCredential = 'dockerhub'
    dockerImage = ''
}

The other important element this is the environment definition. I have defined the registry name as “docker.io/jagadesh1982/sample” which means when we create an image with the final artifact (jar), the image name will be “docker.io/jagadesh1982/sample:<version>. This is very important if you want to push the images to the Dockerhub. Dockerhub expects to have the image name with “docker.io/<user Name>/<Image Name>” in order to upload.

Once the building of the image is done, it is then uploaded to the DockerHub and then removed from the Jenkins Docker container.

Dockerfile - This repo also contains a Dockerfile which will be used to create a Docker image with the final artifact. This means it copies the my-app-1.0-SNAPSHOT.jar to the Docker image. It also run the container from image. The contents looks like this:

FROM alpine:3.2
RUN apk --update add openjdk7-jre
CMD ["/usr/bin/java", "-version"]
COPY /target/my-app-1.0-SNAPSHOT.jar   /
CMD /usr/bin/java -jar /my-app-1.0-SNAPSHOT.jar