Dockerizing Jenkins: Securing Passwords With docker-compose, docker-secret and Jenkins Credentials Plugin

This is the third part of Dockerizing Jenkins series. You can find the previous parts here:

Dockerizing Jenkins 2, Part 1: Declarative Build Pipeline With SonarQube Analysis
Dockerizing Jenkins 2, Part 2: Deployment With Maven and JFrog Artifactory

In this part we will look at:

1. How to use docker-compose to run containers.
2. How to use passwords in docker environment with docker-secrets.
3. How to hide sensitive information in Jenkins with credentials plugin.

In Part 1, we created a basic Jenkins Docker image in order to run a Java Maven pipeline with test and SonarQube analysis. In Part 2, we looked at how to perform deployment using the Maven settings file. As you remember, we saved the password in the file without any encryption, which is not something you would ever do, of course.

All the code for this and the previous parts is in my GitHub repo, and I decided to create a branch for every part, as the master branch will change with every part and older articles would refer to the wrong code base. For this part, the code will be in the branch “dockerizing_jenkins_part_3_docker_compose_docker_secret_credentials_plugin” and you can run the below command to check it out:

git clone https://github.com/kenych/dockerizing-jenkins && \
    cd dockerizing-jenkins && \
    git checkout dockerizing_jenkins_part_3_docker_compose_docker_secret_credentials_plugin 

In this part, we will remove the password from the source code and let the credentials plugin apply credentials to the Config File Provider Plugin. But before changing any code, we will need to switch to using docker-compose instead of using the docker run command. This will give us a chance to leverage the docker secrets feature, along with many other features which you will love.

I updated the runall.sh script, which we used in the two parts before, and replaced it with the docker-compose and download.sh script, which will just download the minimum stuff we will need in advance. I also removed Java 7 and Java 8 installation in favor of using embedded Java 8 from the Jenkins container, as otherwise our download script takes too long and Java comes for free in the image anyway. You can check it later once our Jenkins container is running.

If you were following part one and two, you should know how to pick up the specific Java version anyway using the Maven tool mechanism, and if you want to play with that, just uncomment these lines in the download script, java.groovy ,and in the pipeline as well. Now let’s run the download to make sure we have everything we need:

➜  ./download.sh
2.60.1: Pulling from library/jenkins
Digest: sha256:fa62fcebeab220e7545d1791e6eea6759b4c3bdba246dd839289f2b28b653e72
Status: Image is up to date for jenkins:2.60.1
6.3.1: Pulling from library/sonarqube
Digest: sha256:d5f7bb8aecaa46da054bf28d111e5a27f1378188b427db64cc9fb392e1a8d80a
Status: Image is up to date for sonarqube:6.3.1
5.4.4: Pulling from jfrog/artifactory-oss
Digest: sha256:404a3f0bfdfa0108159575ef74ffd4afaff349b856966ddc49f6401cd2f20d7d
Status: Image is up to date for docker.bintray.io/jfrog/artifactory-oss:5.4.4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8334k  100 8334k    0     0   445k      0  0:00:18  0:00:18 --:--:--  444k

Please note that if you haven’t ever downloaded the images, it will take some time. Now, while it is downloading the stuff we need, let’s look at docker-compose.yml:

version: "3.1"

services:
  myjenkins:
    build:
      context: .
    image: myjenkins
    ports:
     - "8080:8080"
    depends_on:
     - mysonar
     - artifactory
    links:
      - mysonar
      - artifactory
    volumes:
     - "./jobs:/var/jenkins_home/jobs/"
     - "./m2deps:/var/jenkins_home/.m2/repository/"
     - "./downloads:/var/jenkins_home/downloads"
    secrets:
     - artifactoryPassword
  mysonar:
    image: sonarqube:6.3.1
    ports:
     - "9000"
  artifactory:
    image: docker.bintray.io/jfrog/artifactory-oss:5.4.4
    ports:
     - "8081"

secrets:
  artifactoryPassword:
    file: ./secrets/artifactoryPassword

If you were curious, you would ask, why did I call the file docker-compose.yml?
Well, this is a convenient way, as otherwise you can’t call docker-compose commands without the explicit file argument -f, otherwise you would get this error:

docker-compose up
ERROR:
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?
        Supported filenames: docker-compose.yml, docker-compose.yaml

Now let’s investigate the compose file. You may have noticed the “services” section; that is where all containers go. Because we are going to build our own Jenkins image, we added a “build” section to it.

Next, “depends on” will wait for other services before running the Jenkins container.

Then “links” will make it possible to refer to other containers by service name from within the container. This is really cool as we don’t need to define in our pom file the dynamic IP address for the artifactory URL anymore; instead, we can just write http://artifactory:8081/artifactory/example-repo-local and Jenkins will be able to resolve “artifactory” to its IP address.

The very interesting part is secrets. It will bind mount docker secret files, to which we can later refer from container by “/run/secrets/secret_name_here” path. You may ask, couldn’t we simply bind mount just a password file to refer to from within the Groovy script? Well, we could, but best practices require referring to sensitive password information through docker secrets (in a Swarm environment, and here is why).

In this tutorial, we won’t use Swarm where you could create your password through “swarm init” and “echo “your_password_here” | docker secret create artifactoryPassword – “ but instead use docker-compose. So I created the file “artifactoryPassword” under the secrets folder without the password for artifactory in it; instead, it says “write your password here!” This is to show that you should never save the actual password in the repo. So please update it with the actual password, which is “password,” the default password for JFrog artifactory.

Now we are ready to run the containers and later check if the secret file is there.

docker-compose up
Creating network "dockerizingjenkinspart2_default" with the default driver
Building myjenkins
Step 1/6 : FROM jenkins:2.60.1
 ---> f426a52bafa9
Step 2/6 : MAINTAINER Kayan Azimov
 ---> Using cache
 ---> 760e7bb0f335
Step 3/6 : ENV JAVA_OPTS "-Djenkins.install.runSetupWizard=false"
 ---> Using cache
 ---> e3dbac0834cd
Step 4/6 : COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
 ---> Using cache
 ---> 76193d716609
Step 5/6 : RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt
 ---> Using cache
 ---> 2cbf4376a0a9
Step 6/6 : COPY groovy/* /usr/share/jenkins/ref/init.groovy.d/
 ---> 32c36863caef
Removing intermediate container 7d0a005ecf02
Successfully built 32c36863caef
Successfully tagged myjenkins:latest
WARNING: Image for service myjenkins was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.

Pay attention to this warning: “Image for service myjenkins was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up –build`.”

If you don’t get it, that means you have already got that image from the previous tutorial. If this happens, run ”docker-compose build –no-cache“ first and then “docker-compose up” so it recreates a new updated Jenkins image from this source code. Otherwise, Docker will try to use the cache by figuring out if anything changed and hoping that the COPY command in the Dockerfile could be deterministic. Well, perhaps it can, if the hash of the files is used. But I ran into a problem with that for some reason, as Groovy files, which are copied by “COPY groovy/* /usr/share/jenkins/ref/init.groovy.d/
“, were just used from the cache, even after changing them, awkward! But again, if you ever suspect Docker not picking up the updated source code, just use –no-cache.

Now let’s look at the Jenkins’ container secret’s file location :

docker exec -it dockerizingjenkinspart2_myjenkins_1 cat /run/secrets/artifactoryPassword

And you should see the password for Artifactory.

Let’s make sure we have the default Java installation we mentioned before:

docker exec -it dockerizingjenkins_myjenkins_1 java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

Time to dockerize Jenkins with the credentials plugin. We will need to add another Groovy script for saving our credentials:

import jenkins.model.*
import com.cloudbees.plugins.credentials.*
import com.cloudbees.plugins.credentials.common.*
import com.cloudbees.plugins.credentials.domains.*
import com.cloudbees.plugins.credentials.impl.*
import com.cloudbees.jenkins.plugins.sshcredentials.impl.*

println("Setting credentials")

def domain = Domain.global()
def store = Jenkins.instance.getExtensionList('com.cloudbees.plugins.credentials.SystemCredentialsProvider')[0].getStore()

def artifactoryPassword = new File("/run/secrets/artifactoryPassword").text.trim()

def credentials=['username':'admin', 'password':artifactoryPassword, 'description':'Irtifactory OSS Credentials']
def user = new UsernamePasswordCredentialsImpl(CredentialsScope.GLOBAL, 'artifactoryCredentials', credentials.description, credentials.username, credentials.password)

store.addCredentials(domain, user)

As you can see, instead of having the password in the source code, we are reading the password from “/run/secrets/artifactoryPassword.”

Once credentials is created, we can refer to it from Config File Provider Plugin. Amend
mvn_settings from the previous part, as follow:

import jenkins.model.*
import org.jenkinsci.plugins.configfiles.maven.*
import org.jenkinsci.plugins.configfiles.maven.security.*

def configStore = Jenkins.instance.getExtensionList('org.jenkinsci.plugins.configfiles.GlobalConfigFiles')[0]

println("Setting maven settings xml")
def serverCreds = new ArrayList()


//server id as in your pom file
def serverId = 'artifactory'

//credentialId from credentials.groovy
def credentialId = 'artifactoryCredentials'

serverCredentialMapping = new ServerCredentialMapping(serverId, credentialId)
serverCreds.add(serverCredentialMapping)


def configId =  'our_settings'
def configName = 'myMavenConfig for jenkins automation example'
def configComment = 'Global Maven Settings'
def configContent  = '''<settings>
<!-- your maven settings goes here -->
</settings>'''


def globalConfig = new GlobalMavenSettingsConfig(configId, configName, configComment, configContent, true, serverCreds)
configStore.save(globalConfig)

println("maven settings complete")

You may notice we deleted the server section with the password from settings; this is because, in combination with the credentials plugin information, we can now generate it on the fly and apply it to settings. Cool, isn’t it?

Now let's rebuild our image. Remember the warning from docker-compose?

docker-compose up --build

Check that the credentials are created:

And settings now doesn’t have any sensitive information:

You can make the security guys at your company happy now!

Before running our pipeline, remember that “links” in the docker-compose file has removed the necessity of having IP addresses written in the pom file. You can now remove IP addresses from the pom file in the project:

The same is true for the pipeline:

If you are using the code from Part 2, make sure you switch to the “*/dockerizing_jenkins_part_3” branch in the Git repo of the pipeline for maze-explorer project, which has the pom and pipeline changes in it:

And now we have more tidy pipeline:

pipeline {
    agent any

    tools {
/**      Uncomment if want to have specific java versions installed, otherwise maven tool will use jenkins default embedded java 8
 *       you will also need to uncomment java related stuff in java.groovy from dockerize jenkins project and make sure you have these java versions
 *       in your download folder
 */
//        jdk 'jdk8'
        maven 'maven3'
    }

    stages {
        stage('install and sonar parallel') {
            steps {
                parallel(
                        install: {
                            sh "mvn -U clean test cobertura:cobertura -Dcobertura.report.format=xml"
                        },
                        sonar: {
                            sh "mvn sonar:sonar"
                        }
                )
            }
            post {
                always {
                    junit '**/target/*-reports/TEST-*.xml'
                    step([$class: 'CoberturaPublisher', coberturaReportFile: 'target/site/cobertura/coverage.xml'])
                }
            }
        }
        stage ('deploy'){
            steps{
                configFileProvider([configFile(fileId: 'our_settings', variable: 'SETTINGS')]) {
                    sh "mvn -s $SETTINGS deploy -DskipTests"
                }
            }
        }
    }
}

Run the build and cross your fingers for a green, successful build!

That is it, now you have implemented another cool feature and have taken Jenkins dockerization to the next, more secure level.

All steps are coded in the repo below; you can check out and run everything with a single command:

git clone https://github.com/kenych/dockerizing-jenkins && \
   cd dockerizing-jenkins && \
   git checkout dockerizing_jenkins_part_3_docker_compose_docker_secret_credentials_plugin && \
   ./runall.sh