Are You Using Jenkins the Right Way?

Jenkins is a widely used tool, but not everyone uses it in the same way. While some enjoy its test automation abilities, others take advantage of its full automating capabilities, from the code commit through the build, test and up to  deployment. These advanced users are fulfilling the complete continuous integration (CI) vision, which thoroughly facilitates and improves development processes.

How Do You Use Jenkins?

1. Jenkins as a Load Test Automator

Many people use Jenkins as a means to automate and simplify test launching. Instead of manually starting a bunch of scripts, prepare an environment and create an automation in Jenkins. These users don’t think of what they’re doing as part of the CI process, but rather as an easier way of running tests.

While this approach speeds the overall work of the company, we recommend taking advantage of all of Jenkins abilities, to make the whole development process more efficient,  improve the product’s quality and shorten the time to release.

2. Jenkins as Part of the CI Process

Developers who use Jenkins to trigger jobs by commits, are taking advantage of all of Jenkins capabilities. There are two ways to do this:

A. The “Spotless House” Approach

In this approach, performance testing is always included in the full testing process. While this might delay build time, as load tests might take a while to be completed compared to the other tests, the result is assured to be of very high quality. Think of this as cleaning your house from top to bottom every day, including sweeping, washing, doing your dishes and scrubbing windows.

B. The “Clean Enough” Approach

In this approach, load testing is performed as part of the downstream job, but without blocking the main CI job. The full, longer, test can be run once in a decided upon time (let’s say once a week) instead of once a day. This enables you to use Jenkins without pausing your work for too long and waiting 1.5 - 2 hours each time. Think of it as doing the dishes and sweeping the floor every day, and cleaning thoroughly once a week.

3. Jenkins With Taurus — Easier Test Config Maintenance

Test config is an important part of load testing, but it can be hard to maintain. Locally, you will have to manually run pre/post test actions. On Jenkins, there is no version control on the “execute shell” section, resulting in lost changes, complicated and clumsy files management or the need to duplicate the work you have already done on your local computer.

This is where Taurus comes in. Taurus facilitates CI style usage, because you can manage all pre/post test actions in one configuration file as part of version control repository. As a result, all changes are saved in one place, and every change in the test config file can automatically trigger Jenkins to run the test with the changes you made. This allows you to work gradually, improve gradually and apply all changes in one place.

Let’s look at this Taurus test config file:

---

services:


- module: shellexec

  prepare: 

  - ssh ubuntu@54.242.114.239 -i keys/a.pem

  - bash startAgent.sh

  - wget http://www.openss7.org/repos/csv_files/hosts.csv

  - sort -R hosts.csv > random_hosts.csv

  shutdown:

  - echo "shutdown time is $(date)" >> tmp/bla.txt

  post-process:

  - echo "this is a post process" >> tmp/bla.txt

  - mail -a /taurus/bzt.log -s “my_taurus_log_file“ user@example.com < tmp/bla.txt


- module: monitoring

  server-agent:

  - address: 54.242.114.239

    metrics:

    - cpu

    - disks

    - memory


- module: passfail

  criteria:

  - class: bzt.modules.monitoring.MonitoringCriteria

    My Message: CPU on local server>20 for 7s, stop as failed

    subject: 54.242.114.239/cpu

This test config has three modules: shellexec, monitoring and passfail.

The shellexec module has commands for three phases:

• Preparation — For example, log into a remote machine and run a certain process on it, download a data file and manipulate it to use on your test

• Shutdown — A Command that saves the shutdown time to a file

• Post-process — For example, email the log file of the test after it finished running

The monitoring module collects health stats from the computer running Taurus. In this case - CPU, disks and memory.

The pass/fail module determines the success of a load test based on runtime criteria. In this case, if the CPU is higher than 20% for 7 seconds, the test stops. Criteria can be changed easily.

Don’t give up on continuous integration and automating your work. For more information on Jenkins, see our free Webinar "Automated Performance Tests in Jenkins CI Environments".

For more information on Taurus, see the Taurus Knowledge Base.