Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Deployment Pipeline Using Docker, Jenkins, Java, and Couchbase

DZone's Guide to

Deployment Pipeline Using Docker, Jenkins, Java, and Couchbase

Learn how to create a deployment pipeline using Jenkins and Docker for a Java application talking to a database.

· Integration Zone
Free Resource

Modernize your application architectures with microservices and APIs with best practices from this free virtual summit series. Brought to you in partnership with CA Technologies.

This blog explains how to create a deployment pipeline using Jenkins and Docker for a Java application talking to a database.

Jenkins support the creation of pipelines. They are built with simple text scripts that use a Pipeline DSL (domain-specific language) based on the Groovy programming language.

The script, typically called Jenkinsfile, defines multiple steps to execute both simple and complex tasks according to the parameters that you establish. Once created, pipelines can build code and orchestrate the work required to drive applications from commit to delivery.

A pipeline consists of steps, node, and stage. A pipeline is executed on a node — a computer that is part of Jenkins installation. A pipeline often consists of multiple stages. A stage consists of multiple steps. Read Getting Started with Pipeline for more details.

For our application, here is the basic flow:

docker-pipeline-jenkins

Complete source code for the application used is at github.com/arun-gupta/docker-jenkins-pipeline.

The application is defined in the webapp directory. It opens a connection to the Couchbase database and stores a simple JSON document using Couchbase Java SDK. The application also has a test that verifies that the database indeed contains the document that was persisted.

Many thanks to @alexsotob for helping me with Jenkins configuration.

Let’s get started!

Download and Install Jenkins

Download Jenkins from jenkins.io. This was tested with Jenkins 2.21.

  • Start Jenkins:
  • JENKINS_HOME=~/.jenkins java -jar ~/Downloads/jenkins-2.21.war --httpPort=9090
    This command starts Jenkins by specifying the home directory where all the configuration information is stored. It also defines the port on which Jenkins is listening, 9090 in this case.
  • First, start of Jenkins shows the following message in the console:
    *************************************************************
    *************************************************************
    *************************************************************
     
    Jenkins initial setup is required. An admin user has been created and a password generated.
    Please use the following password to proceed to installation:
     
    3521fbc3d40448efa8942f8e464b2dd9
     
    This may also be found at: /Users/arungupta/.jenkins/secrets/initialAdminPassword
     
    *************************************************************
    *************************************************************
    *************************************************************
    Copy the password shown here. This will be used to unlock Jenkins.
  • Access the Jenkins console at localhost:9090 and paste the password:docker-pipeline-jenkins-unlockClick on Next.
  • Create the first admin user as shown:
    docker-pipeline-jenkins-create-admin-user
    Click on Save and Finish.
  • Click on Install suggested plugins:docker-pipeline-jenkins-install-suggested-plugins
    A bunch of default plugins are installed:docker-pipeline-jenkins-installing-suggested-plugins
    (I found it surprising that Ant and Subversion are the default plugins.)
  • The login screen is prompted.
    docker-pipeline-jenkins-login
    Enter the username and password specified earlier.
  • Finally, Jenkins is ready to use:
    docker-pipeline-jenkins-start-using
  • That’s quite a bit of steps to get started with basic Jenkins. Do I really have to jump through all these hoops to get started with Jenkins? Is there an easier, simpler, dumber, lazier way to start Jenkins? Follow Convention-over-Configuration and give me one-click pre-configured installation.

    Install Jenkins Plugins

  • Install the required plugins in Jenkins.

    1. If your Java project is built using Maven, then you need to configure Maven in Jenkins. Click on Manage Jenkins, Global Tool Configuration, Maven installations, and specify the location of Maven.docker-pipeline-jenkins-configure-maven
      Name the tool as Maven3 as that is the name used in the configuration later.Again a bit lame, why can’t Jenkins pick up the default location of Maven instead of expecting the user to specify a location.
    2. Click on Manage Jenkins, Manage Plugins, Available tab, search for docker pipe. Select CloudBees Docker Pipeline, click on Install without restart.
      docker-pipeline-jenkins-pipeline-plugin
      Click on Install without restart.Docker Pipeline Plugin plugin understands the Jenkinsfile and executes the commands listed there.
    3. The next screen shows the list of plugins that are installed:docker-pipeline-jenkins-pipeline-plugin-restart-jenkins
      The last line shows that CloudBees Docker Pipeline plugin is installed successfully. Select Restart Jenkins checkbox. This will install restart Jenkins as well.

    Create Jenkins Job

    Let’s create a job in Jenkins that will run the pipeline.

    1. After Jenkins restarts, it shows the login screen. Enter the username and password created earlier. This brings you back to Installing Plugins/Upgrades page. Click on the Jenkins icon in the top left corner to see the main dashboard:docker-pipeline-jenkins-dashboard
    2. Click on create new jobs, give the name as docker-jenkins-pipeline and choose the type as Pipeline:docker-pipeline-jenkins-create-projectClick on OK.
    3. Configure Pipeline as shown:
      docker-pipeline-jenkins-configure-pipelineLocal git repo is used in this case. You can certainly choose a repo hosted on github. Further, this repo can be configured with a git hook or poll at a constant interval to trigger the pipeline.Click on Save to save the configuration.

    Run Jenkins Build

    Before you start the job, Couchbase database need to be explicitly started as:

    docker run -d --name db -p 8091-8093:8091-8093 -p 11210:11210 arungupta/oreilly-couchbase:latest


    This will be resolved after #9 is fixed. Make sure you can access Couchbase at http://localhost:8091, use Administrator as the login and password as the password. Click on Data Buckets tab and see the books bucket created.

    docker-pipeline-couchbase-books

    Click on Build Now and you should see an output similar to:


    docker-pipeline-jenkins-build-run

    All green is good!

    Let’s try to understand what happened behind the scenes.

    Jenkinsfile describes how the pipeline is built. At the top level, it has four stages — Package, Create Docker Image, Run Application, and Run Tests. Each stage is shown as a box in Jenkins dashboard. Total time taken for each stage is shown in the box.

    Let’s understand what happens in each stage.

    Package: Application source code lives in the webapp directory. The Maven command:

    mvn clean package -DskipTests

    is used to create a JAR file of the application. Note that the Maven project also includes the tests and are explicitly skipped using:

    -DskipTests

    Typically, tests would be in a separate downstream project. Maven project creates a far JAR file of the application and includes all the dependencies.Create Docker Image – Docker image of the application is built using the Dockerfile in the webapp directory. The image simply includes the fat JAR and runs it using:

    java -jar


    Each image is tagged with the build number using:

    ${env.BUILD_NUMBER}


    Run Application: Running the application involves running the application Docker container. The IP address of the database container is identified using the command:

    docker inspect


    The database container and the application container are both running in the default bridge network. This allows the two containers to communicate with each other. Another enhancement would be to run the pipeline in a swarm mode cluster. This would require to create and use an overlay network.

    Run Tests: Tests are run against the container using the command:

    mvn test


    If the tests pass, the image is pushed to Docker Hub. The test results are captured either way. This stage also shows the usage of the try/catch/finally block in the Jenkinsfile. If the tests pass, then the image is pushed to Docker Hub. In this case, it is available here.

    TODOs…

    • Move the tests to a downstream project (#7).
    • Use Git hook or poll to trigger pipeline (#8).
    • Automate database startup/shutdown (#9).
    • Run pipeline in a cluster of Docker Engines with Swarm mode (#10).
    • Show alternate configuration to push image to bintray (#11).
    • Another pain point is that global variables syntax does not seem to be documented anywhere. It is only available at <JENKINS-HOST>:<JENKINS-PORT>/job/docker-jenkins-pipeline/pipeline-syntax/globals. This is, again, slightly lame!

    • not impossible, just not implemented yet” #sadpanda.

    The Integration Zone is proudly sponsored by CA Technologies. Learn from expert microservices and API presentations at the Modernizing Application Architectures Virtual Summit Series.

    Topics:
    deployment ,groovy ,pipeline ,jenkins ,deployment pipeline ,scripts ,programming

    Published at DZone with permission of Arun Gupta, DZone MVB. See the original article here.

    Opinions expressed by DZone contributors are their own.

    THE DZONE NEWSLETTER

    Dev Resources & Solutions Straight to Your Inbox

    Thanks for subscribing!

    Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

    X

    {{ parent.title || parent.header.title}}

    {{ parent.tldr }}

    {{ parent.urlSource.name }}