{{announcement.body}}
{{announcement.title}}

Run a Jenkins Instance From Code

DZone 's Guide to

Run a Jenkins Instance From Code

Learn how you can use the application of DevOps processes in building a Jenkins instance from code.

· DevOps Zone ·
Free Resource

building

Learn how to construct a Jenkins Instance using the DevOps process.

It has been some time since I started working with Jenkins. Some of my daily duties with this tool go from supervising the execution of legacy jobs to maintaining and enhancing declarative multibranch pipelines, always trying to achieve continuous integration and continuous delivery practices to promote packaged software through many environments until it gets the client’s hands.

Reaching such a level of automation involves time, teams and processes. From my experience, it’s a task that requires a lot of time and effort, and everyone involved in the software development and delivery process has to take up this new way of thinking and working, and adapt to it.

After spreading a few of my thoughts on the DevOps culture, I would like to focus once again on the Jenkins topic. I spend most of my time between environments and for each environment, I work on an entirely different Jenkins. I test new features from fancy plugins that improve and clarify the software delivery process on a development environment. Once I verify the new feature works well, I spend more time promoting it to the rest of the environments. This sounds like a repetitive task, but actually I tend to avoid these kinds of tasks, for over the years I pursue the adoption of EaC, Everything as Code, but for some reason, I had no chance to apply it yet on the Jenkins installation scope.

Goal

The desired state I would like to reach is to be able to build a Jenkins instance where all of its configuration and job definitions goes into declarative files. This way we have immutable Jenkins instances that deploy on any environment, whenever you add a plugin or job, or modify a configuration, it gets stored and versioned on Git. This way, no unknown configuration is able to break the desired state of the instances anymore and anyone can deploy the exact state on their localhost, for testing purposes, for example.

Initial Steps

I have decided to structure the project into two repositories.

There’s an initial project to set up the configuration at the Jenkins instance level. On a file, we add the plugins to install and lock their versions. One of the plugins, Jenkins configuration as code lets us predefine the global configuration of Jenkins and the rest of the plugins without interacting with the Jenkins UI, finally a Dockerfile is used to build a Docker image that includes all the initialization files.

A second project is used to interact with the Jenkins DSL plugin; the goal is to store the jobs, folders, and views as code on groovy files, so only what we have defined on our Git repository applies to the Jenkins instance.

Docker

A widely known implementation of the container software abstraction is used to pack, gather dependencies, and automate the deployment. We are going to build the Jenkins instance using a Dockerfile.

FROM jenkins/jenkins:lts

COPY init-scripts /usr/share/jenkins/ref/init.groovy.d

ADD plugins.txt /usr/share/jenkins/ref/
ADD jenkins.yaml /usr/share/jenkins/ref/

RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt
ENV CASC_JENKINS_CONFIG /usr/share/jenkins/ref/

VOLUME /var/jenkins_home


The following shell script can be a useful resource to launch the build phase.

#!/usr/bin/env bash

docker build -t jenkins_as_code:0.1.0 .

There are two phases, the one that builds the container image and the phase that runs it.

Build Phase

Plugins Installation

To automate the installation of the desired plugins, we are going to make use of an existing shell script included in Jenkins. The script its located at /usr/local/bin/, inside the container, with the name install-plugins.sh, to install the plugins save them inside a file and call the script with the file as a parameter.

workflow-aggregator:2.6
configuration-as-code:1.30
configuration-as-code-support:1.18
credentials:2.3.0
blueocean:1.19.0
job-dsl:1.74

Now we can store the plugins that need to be installed and lock their versions.

JCaC Jenkins Plugin

Working on Jenkins involves many plugin installations, and for each plugin a specific configuration must be set up. All these configuration changes add up over time; a Jenkins instance tends to evolve based on the needs of the software delivery process.

The GUI is used to configure Jenkins; the thing is that there is no place to centralize and store what configurations are applied other than browsing it through the web.

Most of the people have been solving this issue using Groovy initialization scripts. It works, but as it involves some coding, its not as friendly and readable as desired.

The good news is the existence of a plugin that lets you define the global and plugin configurations through a YAML file. It’s great because it handles the configuration changes and we can deploy an initial Jenkins install with all the configurations already applied.

So we can progress on the primary goal, the deployment of immutable Jenkins instances is defined by versioned configuration files.

This plugin even lets you define a seed job, its a job that fetches DSL files and applies the changes creating other Jobs, views, and many more features.

In our case, the following jenkins.yaml file is used to provide the configuration. The seed job fetches another project from Github that contains a few DSL based objects to create resources.

---
jenkins:
  systemMessage: "Jenkins As Code Concept."
  views:
    - myView:
        name: "Jobs Config as Code"
security:  
  globalJobDslSecurityConfiguration:    
    useScriptSecurity: false
jobs:  
  - script: >      
      freeStyleJob("Jobs Generator") {        
        scm {
          github('imanol-dev/jenkins_as_code_jobs', 'master')        
        }
        steps {
          dsl {
            external('*.groovy')
          }
        }
      }

A great thing about this plugin is that you can modify the YAML file and reload the new configuration, applying all the changes without the need to recreate the Jenkins instance.

Runtime Phase

Default User

While working on this, I faced an issue trying to automate the creation of a default administrator user through a configuration file.

To solve it, I did some research and found a repository that makes use of Groovy initialization scripts; the script goes inside the container in the /usr/share/jenkins/ref/init.groovy.d directory.

This way, to create the user, we only have to fill the following environment variables:

ADMIN_USERNAME=
ADMIN_PASSWORD=

Skip the Wizard

Every time you deploy a new Jenkins instance, the first time you log in, you get prompted with a Configuration Wizard that helps you set up a few initial parameters and install some suggested plugins.

For newcomers, this can be kind of helpful, but we are trying to achieve automation that deploys what is written in the configuration files, so we don’t need this feature.

The way to avoid it is to run the container with the following environment variable.

JAVA_OPTS=”-Djenkins.install.runSetupWizard=false”

Get It Running

I work on a Linux based OS; it makes it easier for me to get things done, if you’re not already on a Unix based OS I suggest you think about the change.

To get this working, you need Bash and Docker. Once you satisfy these requirements, you can clone the project.

.
├── init-scripts
│ └── admin-user.groovy
├── config
│ └── runtime.env
├── Dockerfile
├── jenkins.yaml
├── plugins.txt
├── build
└── run

The first step is building the Docker image from the current directory. There’s a bash script to do this named build. It’s a basic Docker build command to generate the jenkins_as_code image in your local registry.

The Jenkins instance can be accessed through a web browser on the following localhost:8080 URL, sign in using the credentials initially set on the config/runtime.env file.

Once inside, there’s a seed job to generate, through the DSL plugin, the rest of the desired jobs.

Job DSL Plugin and Job Definition

I have found a few ways to create Jenkins jobs; you can create them manually through the web interface, define them on XML files using the Jenkins CLI to import them or make use of the DSL plugin.

The DSL plugin lets you define jobs on a declarative form, more human-readable.

I have defined a few jobs on a separate repository, using the seed job, I reference this repository, so after executing it, all the new jobs appear automatically. The seed job can be triggered off a git change, so if you delete or modify a job through code, this gets updated on the Jenkins instance without manual execution.

If you have the Jenkins instance running, roll the seed job to create the new Jenkins objects.

What About Ansible?

I did some research about this matter solved with Ansible, and I’ve found a few roles that let you install Jenkins in remote machines through Ansible Roles. There’s a role that caught my attention that lets you tune up the configurations, setting up the variables associated to the role, this approach lets you choose between Containerized or VM deployments and define as code all your Jenkins Jobs in an XML format.

From my point of view this role its a great option but I wrote this post to easily store all your Jenkins Configurations and Job definitions as code in Docker containers, I am planning to evolve this first approach to end deploying a Jenkins cluster in Kubernetes using Helm so, from my point of view, Ansible does not fit for what I initially planned.

Conclusions

There are plenty of better approaches for the multi-environment deployment matter. A single Jenkins across all the environments could do the job, but sometimes it’s not only about the best technical solution.

In my case, sometimes, I have to adapt to entirely isolated environments, with independent data centers and different network settings for the same project. A few years already in the business world have taught me that there’s probably something more significant than my ideas already running and I have to adapt to it.

I enjoyed this research; I wanted to find a way to automate a few repetitive and manual tasks so I could deploy multiple Jenkins instances with the same configurations but with different Jobs.

Thanks for reading and if in some manner, this is helpful for you, do not hesitate writing me.

Further Reading 

Top 5 Jenkins Plugins

Dockerizing Jenkins 2, Part 1: Declarative Build

Topics:
jenkins configuration as code ,jenkins installation ,configuration as code ,devops ,automation ,docker

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}