Tips on Jenkins — a Decade Later — for Continuous Delivery

My first encounter with Jenkins (actually, Hudson — Jenkins was forked from Hudson many years ago) was almost a decade ago when I had to use it for Continuous Integration (CI). A few days ago, I had to work with it again. This time, it was for the task of Continuous Delivery (CD). Jenkins has undergone many changes over the years and seems to have evolved into quite a powerful tool with a lot of plugins.

My task was quite simple. I had to download a package in ZIP format from a repository and install it on a specific node. For this, I did the following:

1. Used the Pipeline plugin

2. Defined a node in Jenkins such that I could use it in the script

3. Defined the actual steps of the task in the script portion of the pipeline

The script did the following:

1. Download the package from the specified repository

2. Unzip the package into the proper directory

3. Step into the "bin" directory and grant execute permissions to the common scripts

4. Step into the "usecase" directory and grant execute permissions to the 'load' and 'analysis' scripts

5. Step into the config directory and update the paths in the config file

I wrote all the steps in the "script" portion of the Pipeline plugin and executed. After a couple of trials to sort out usage of the proper set of quotes, executing all the steps in a single shell, spelling mistakes, etc, I got the script working. The script looked something like below:

node {
    name {
        'node-1'
    }

    stage('Install') {
        sh '''
            echo "Downloading package"
            wget url-of-repository/package.zip
            echo "Unzipping package"
            unzip package.zip
            echo "Setting up the package"
            cd bin
            chmod +x common.sh
            chmod usecase
            chmod +x load.sh analysis.sh
            echo "Setup complete"
        '''
    }
}

The very next day, I realized the folly of this script. Due to the fact that the script was written as part of the Jenkins job defintion, I had very little control over it. Additionally, I was at the mercy of the author. Hence, I decided to make a simple change.

I defined a setup.sh script and deployed it as part of the installation package. I moved all the application initialization steps into the setup script. After doing so, the script looked something like below:

node {
    name {
        'node-1'
    }

    stage('Install') {
        sh '''
            echo "Downloading package"
            wget url-of-repository/package.zip
            echo "Unzipping package"
            unzip package.zip
            echo "Setting up the package"
            cd usecase
            chmod +x setup.sh
            . ./setup.sh /home/usecase
            echo "Setup complete"
        '''
    }
}

and setup.sh was like below

#!/bin/bash
APP_INSTALL_DIR=${1}

cd ${APP_INSTALL_DIR}
cd bin
chmod +x common.sh
chmod usecase
chmod +x load.sh analysis.sh

A Tip

To make the package flexible, I had used placeholder variables in many places. As part of the setup, it updates the config files and replaces the placeholder pattern by taking values passed with the Pipeline script (as it was defined to be having parameters). Initially, the pattern replacement operation did not work as expected. As I was using 'sed' for replacement it was not an unknown task. On using the proper debug statements, I realized by mistake. As I was replacing paths, I had the forward slash (/) character in the string to be used as a replacement. If this string is used as is, it trips up 'sed', because the actual line executed becomes

 sed -e 's/APP_PATH_TO_REPLACE//home/usecase/g' > config.txt

 When 'sed' encounters '//', it trips as it believes no replacement pattern has been specified.

To overcome this problem, I had to specify the value of the variable APP_PATH_TO_REPLACE as '\/home\/usecase' instead of '/home/usecase', so the 'sed' command becomes