Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Automated Jenkins Configuration via Cloud Scripting Add-On

DZone's Guide to

Automated Jenkins Configuration via Cloud Scripting Add-On

Using Jelastic Private Cloud and cloud scripting options, learn how to simplify the implementation and automation of Jenkins in your cloud environment.

· DevOps Zone
Free Resource

Download “The DevOps Journey - From Waterfall to Continuous Delivery” to learn learn about the importance of integrating automated testing into the DevOps workflow, brought to you in partnership with Sauce Labs.

Evolving the concept of the DevOps approach at the Jelastic Platform, this time we are pleased to present you with the third guide from this devoted series. The new solution, described in today’s article, gives even greater automation through not just ensuring the project lifecycle management, but also simplifying the process of Jenkins continuous integration server configuration.

This is achieved through combining two progressive technologies: Jelastic pluggable add-ons and Cloud Scripting operations – the next generation solution for application and server management within the Cloud, which is currently being intensely developed by the Jelastic team. In particular, such an integration allows to automatically create the complete list of job templates, which just need to be slightly tuned (with operations like setting a URL to the repository with the required project, instead of the default one) to complete the Jenkins server adjustment.

The true efficiency of such an implementation is maximally revealed in the case of handling multiple projects, as instead of the countless step-by-step manual configurations, required for each of the Jenkins servers to be properly set, you’ll need to create a single add-on just once. Subsequently, it can be easily applied to the required environment in a few clicks with only a few user-dependent parameters required to be adjusted. Such an approach saves an increasing amount time on each following installation.

Project-update_2














As this article is tightly connected with the previous two parts of the ”DevOps Tools for Continuous Delivery in Jelastic Private Cloud”series, we will periodically refer to them in order not to repeat describing the same steps. That’s why we recommend you get acquainted with these guides firstly before proceeding further:


General Preparations

First of all, let us remind you of  a few obligatory arrangements, required to be accomplished in order to ensure the best DevOps experience:

  • Your Jelastic installation should include three individual hardware sets allocated for the developmenttesting and production teams separately:

image07








  • Since the core of the continuous application delivery approach we are referring to is represented with the Jenkins server, you need to have an environment with the appropriate software running. As you can remember from our previous guides within the series, getting it becomes fairly easy with the embedded Jelastic Marketplace, which can be accessed either via the appropriate site page or directly through the dashboard.

image00

We’ve considered these steps in details previously, thus we are sure you won’t meet any troubles here.

Configuring Jenkins Using Cloud Scripting

As it was already mentioned within this guide’s introduction, the greatest part of the work at the stage of Jenkins server’s setting up is going to be automated. To gain this, the appropriate configurations should be implemented within the code firstly, which will be subsequently executed at your application server with Jenkins handling. At Jelastic, this could be done through preparing a special JSON manifest file, which, being integrated to the platform, will represent a pluggable add-on, that can be easily run by any user directly through the dashboard.

Creating Add-On

Let’s start with composing the basis for our add-on in order to get the ability to apply it to the required Jelastic environment further:

  1. Log into your JCA panel as an administrator user and navigate to the Marketplace section using the main menu to the left.

image04

Once inside, expand the Add list at the top pane and select the Add-on option.

  1. Within the opened editor, paste the general code template of the following structure:
{
    "jpsType": "update",
    "jpsVersion": "0.3",
    "application": {
     "name": "package_name",
     "logo": "logo_url",
     "type": "java",
     "onInstall": {
         "executeShellCommands": {
             "nodeType": "tomcat7",
             "commands": [
                     custom_code
             ]
         }
     }
    }
}


The highlighted parameters should be substituted with the appropriate data – here you need to define the general information about your package and specify the operations it is intended to perform:

    • package_name – add-on name it will be titled at the users’ dashboard (e.g Jenkins Configurations)
    • logo_url – link to the image that will be used as a package icon (you can use our default one through copying and specifying this URL)
    • custom_code – list of comma-separated shell commands, that will be run on add-on’s appliance

Whilst the first two settings are obvious, the last one needs to be considered in more detail, thus the next section of the guide is devoted to the specifics of the required operations’ scripting.

Composing Script

In our example, the Jenkins integration tool is installed over Tomcat 7, so all the commands used would be based on its specifications. We intend to automate the process through scripting the following operations:

  • Uploading the scripts, required for setting the jobs.
  • Configuring Jenkins itself.
  • Creating and linking jobs to automatically lead an application through the lifecycle.

Further, we’ll examine each of these points in more detail and prepare the core of our manifest according to this flow.

Note: Below we provide you with the example code, which needs to be modified according to the specifics of your application. In order to ensure this was done correctly and all of the actions were scripted properly, we recommend a preliminary test of your code. This could be done through creating a separate environment with the same topology your add-on is being created for (i.e. Tomcat 7 withJenkins app deployed in our case) and running the adjusted commands directly inside your compute node’s container using SSH. If you need any assistance with this, please refer to the complete guidance on the DevOps Add-on for Jenkins Troubleshooting, which additionally includes descriptions of the expected results for each of the presented operations.

So, let’s reveal how the above suggested processes can be easily accomplished, with the help of the appropriate Cloud Scripting options. Subsequently, all of them need to be inserted instead of the custom_code parameter in the manifest example, in the same order they are described below.

  1. Here is the first part of the script we propose to create:
“mkdir /opt/tomcat/demo“,

“cd /opt/tomcat/demo”,

“echo ENV_NAME= > /opt/tomcat/demo/variables”,

“curl -fsSL file_url -o install.sh“,

“curl -fsSL file_url -o transfer.sh“,

“curl -fsSL file_url -o migrate.sh“,

“curl -fsSL file_url -o runtest.sh“,



In this piece of code you can recognize the process of uploading the scripts (they are required for setting Jenkins jobs) to the container. Previously, we’ve done this manually and considered everything in detail, thus here, we’ll just remind you of some crucial points (highlighted in the example above) that you need to pay attention to:

  • /opt/tomcat/demo – the full path to the folder where all your scripts information will be stored (we’ll use demo, but you could create and use any other desired directory – just don’t forget to specify the correct path everywhere throughout your code)
  • curl -fsSL file_url -o file_name – this command initiates file uploading inside the container; for this example we require the following scripts:
    • install.sh – automatically logs in as a specified user and creates a new environment with the topology set according to the Jelasticmanifest file
    • transfer.sh – passes the environment ownership to the specified user based on the environment transferring feature
    • migrate.sh – relocates the environment to the specified hardware set (hardnode group)
    • runtest.sh – in our example, it simulates the testing activities for demonstration purposes, thus we don’t provide its code. If required, create your own one according the specifics of your application and prescribe its uploading alongside the rest of the scripts


Note that before applying, each of the script templates, linked above, have to be additionally adjusted by substituting the following parameters according to your platform’s settings:


  • /path/to/scripts/ – the same path as you’ve set above (/opt/tomcat/demo in our case)
  • {cloud_domain} – your Jelastic Platform domain name
  • {JCA_dashboard_appID} – your dashboard ID, that could be seen within the Platform.DASHBOARD_APPID parameter at the JCA >About section
  • {JCA_appstore_appID} – appstore ID, listed within the same section (the Platform.APPSTORE_APPID parameter)
  • {url_to_manifest} – link to the Jelastic manifest file created according to our documentation (we’ll use the same one as before – it sets up two Tomcat application servers fronted by the NGINX load-balancer)

In addition, the 3rd string defines creation of the variables file alongside the uploaded scripts, intended to be used for further storing the automatically generated name for the environments with your app’s different versions.

  1. The second part of code will remove all the existing (i.e. that are installed automatically during the initial tool deployment) Jenkins plugins in order to prevent incompatibility issues or a mismatch of their versions with the upcoming configurations:
“rm -rf /opt/tomcat/temp/.jenkins/plugins”,


  1. After that, you need to move to the main .jenkins directory and add the necessary plugins and configuration files to the Jenkins integrator instead of the deleted ones by uploading and extracting the appropriate plugins.zip archive.
“cd /opt/tomcat/temp/.jenkins/”,

“curl -fsSL file_url -o plugins.zip“,

“unzip -o plugins.zip >/dev/null 2>&1″,



We recommend to use the package we’ve prepared for this example, which already contains all the necessary modules. However, you can complement it with other plugins if necessary or just pack your own zip archive – in this case, note that there are several obligatory tools it should comprise (namely: SCM API PluginGIT pluginGIT client pluginEnvironment File Plugin) and all of them should be put in theplugins folder inside the archive.

  1. The next section of code uploads and extracts another required package (jobs.zip), that will set Jenkins jobs according to the data inside.
“curl -fsSL file_url -o jobs.zip“,

“unzip -o jobs.zip >/dev/null 2>&1″,



The archive structure should be the following: one general jobs folder, which may include multiple directories, named after the appropriate comprised jobs. Each of such directories should contain the config.xml file (with all the required configs specified inside) and the buildsfolder with an empty legacyIds file. We’ve already prepared such an archive, which contains settings for creation of the jobs, devoted to each step of the proposed application lifecycle flow:

Create Environment > Build and Deploy > Dev Tests > Migrate to QA > QA Tests > Migrate to Production


Note that in case you are using your custom path to the scripts folder, you need to edit the Build and Deploy job in our example package, as it contains the defined location of the file inside that directory:


<filePath>/opt/tomcat/demo/variables</filePath>



Everything else may remain as is, however, you can also edit it or add your own jobs if needed.

5. Finally, define the restart of the server in order to apply all of the executed configs.

“sudo /etc/init.d/tomcat restart >/dev/null 2>&1″


Being combined together, all these code parts will automatically perform almost all the required Jenkins configurations.

Thus, now you can place them within the appropriate commands manifest section as it was suggested above – you’ll get something similar to the following:

image03

Click Save to finish and append your add-on to the list of the already available installations.

Enabling the Add-On

Finally, to make our module accessible via the dashboard, find and select it within the list, click on the Publish button above and confirm this action at the shown pop-up.

image09





Everything is prepared now, so let’s just see how quick and simple the implementation of our DevOps approach becomes now.

New DevOps Experience

Navigate to the Jelastic dashboard (for the account your Jenkins environment is running at) and perform the following to plug your add-on to the appropriate application server.

  1. Hover over the Tomcat 7 node and click the appeared Add-ons icon to display the same-named tab with the list of suitable pluggable modules.

image08

Locate your add-on (named Jenkins configuration in our case) and Install it with the appropriate button.

  1. Wait for the process to be successfully completed and access the Jenkins dashboard (i.e. Open your environment in browser).

image10

As you can see, we’ve got the list of already prepared jobs.

  1. Now you need to slightly adjust each of these actions, namely:
  • specify the link to the repository with your project for the Build and Deploy job  
  • set the correct credentials inside the Execute shell box for the rest of the jobs

In order to access a particular job’s parameters and change them, use the Configure context menu option (can be called through right-clicking) for the required record:

image02













Note: As an example for this guide, we use a 


  • {dev_user}@example.com – login (email address) of the dev user’s account
  • {password} – password for the user, specified above
  • {cloud_domain} – domain name of your Jelastic Platform
  1. The last thing left to perform is to bind the launching of the first job from our cycle (i.e. Create Environment) to the commit action, that occurred at the repository. As you already know from the previous guides, this could be done through the Webhook option – here is a reminder of the workflow (with the GitHub repo settings as an example):

image01

That’s all! Now you have a fully configured and ready-to-work integration complex, that waits only for the project change to be committed in order to automatically create an environment, deploy a new application version and move it through the set of required testing and relocation steps. After that, your checked application will appear at production.

And having the Jelastic attachable add-on for Jenkins configuration already prepared, you can quickly apply this approach to any of your projects through deploying a new continuous delivery mechanism within a few clicks and with slight adjustments done.

Discover how to optimize your DevOps workflows with our cloud-based automated testing infrastructure, brought to you in partnership with Sauce Labs

Topics:
jelastic ,cloud ,devops ,jenkins ,automation

Published at DZone with permission of Tetiana Markova, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}