Over a million developers have joined DZone.

Automating JMeter Test Plans in Jenkins

JMeter is an Open Source tool written in Java to test system performance simulating different types of heavy load on the system under test using different protocols.

Download Forrester’s “Vendor Landscape, Application Performance Management” report that examines the evolving role of APM as a key driver of customer satisfaction and business success, brought to you in partnership with BMC.

JMeter (http://jmeter.apache.org/) is an Open Source tool written in Java to test system performance simulating different types of heavy load on the system under test using different protocols (HTTP, SOAP, JDBC, LDAP, JMS, etc.). This article is specific for people who want to understand how to automate JMeter test plans execution using Jenkins: I am assuming the reader has good knowledge of JMeter and Maven and omitting some JMeter and Maven features details. The JMeter release I am referring to in this post is the 2.12. Test plans in JMeter can be executed by command line or through its Java Swing GUI. Running through Jenkins doesn't require them both. You need to use just Jenkins (of course), Maven and the Performance Plugin (https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin). You can use the JMeter GUI just to make changes to a test plan.

Create the Test Plan

Let's create a simple test plan first. Suppose it is a HTTP test plan. You can start from Web Test Plan, one of the templates available in JMeter. This template creates a HTTP test plan for the Google home page (www.google.com). Add a parameter for the Number of Thread property of the Thread Group:

Image title

execute the test plan to check that everything is fine and then save it (for this example the chosen name is HTTPExampleTestPlan.jmx). As stated before, this is the only step for which you need to use JMeter.

Create the Maven Project

Create a folder for the Maven project and copy the .jmx test plan file into the src/test/jmeter subfolder. Then create the POM file for the project:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">    


You need to write just a basic POM file that includes only the JMeter Maven plugin (http://jmeter.lazerycode.com/). The <configuration> section of the plugin is the JMeter configuration. Here you can tell Maven that the test plan is parameterised and which configuration property is parameterized (in our example the threadCount property only). You can notice that there is no reference to the test plan name in the POM file code. Any JMX file that is saved in the src/test/jmeter subfolder is automatically executed at runtime by the JMeter Maven plugin. You can run this project from a shell command to test it before moving to Jenkins:

mvn jmeter:jmeter -Dperformancetest.threadCount=2

The results of this test will be created in the target/jmeter/report folder.

Install the Performance Plugin

Install the Performance Plugin from the Jenkins Update Center the usual way. This plugin allows capture from JMeter and JUnit reports and generation of charts with the trend report about performance. In this post we will focus on the JMeter part only.

Create the Jenkins Build Job

From the Jenkins Dashboard create a new freestyle build job and make it paremeterised in order to accept the value for the thread count property of the Meter test plan:

Image title

Add a Maven build step. Set the goals the same way as for running the Maven project from a shell command and set the location of the POM file of the project:

Image title

Add then the Performance plugin as post-build action and set the location of the report file inside the Maven project:

Image title

Finally set the other properties (thresholds) for your test plan execution:

Image title


Set the schedule for the build job and execute it. After each build you can see a text summary of the test result in the console output:

[info] Completed Test: HTTPExampleTestPlan.jmx
[info] Building JMeter Report(s)...
[info] Raw results: C:\Guglielmo\Code\JMeter\target\jmeter\report\HTTPExampleTestPlan.jtl
[info] Test report: C:\Guglielmo\Code\JMeter\target\jmeter\report\HTTPExampleTestPlan.jtl-report.html
[INFO] Test Results:
[INFO] Tests Run: 1, Failures: 0, Errors: 0
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.731s
[INFO] Finished at: Tue Dec 09 10:55:37 GMT 2014
[INFO] Final Memory: 7M/15M
[INFO] ------------------------------------------------------------------------
Performance: Percentage of errors greater or equal than 0% sets the build as unstable
Performance: Percentage of errors greater or equal than 0% sets the build as failure

and in a graphical format as well in the project page for the build job:

Image title

Clicking on the Performance Trend link in the left column of the project page

Image title

you can see both the overall and the single test cases performance trend (in our example they overlap because we have a single test case) across the build job executions:

Image title

Clicking on the Performance Report link in the left column of each build page

Image title

you can see the full detailed performance report

Image title

Clicking on the test name in the table you can see the details of the test case for the current run split by thread (the example shown in the following figure refers to a run with THREAD_COUNT = 4):

Image title

Following the simple steps described in this article you can schedule and execute any type of JMeter test plan for any type of protocol through Jenkins.

See Forrester’s Report, “Vendor Landscape, Application Performance Management” to identify the right vendor to help IT deliver better service at a lower cost, brought to you in partnership with BMC.


The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}