Advanced Load Testing Scenarios With JMeter - Part II: Data-Driven Testing and Assertions

Welcome to part 2 of the “Advanced Load Testing with JMeter” series. If you missed part 1, which explained correlations, check it out here.

In this part, we will go over Data-Driven Testing and Assertions.

When we run the same query over and over again, the database doesn’t always build new queries, but reuses the previous ones, to improve speed and optimize performance. This caching obstructs our ability to test our scripts.

To avoid this problem, you can incorporate a CSV file with different variables into the JMeter test. The variables are defined in different columns, and these columns can later be referenced in JMeter.

Here we are using the simple Travel Agency demo we always use. The columns are the origin and destination ports, and the rows are ports’ names.

To incorporate the variables, add a CSV configuration element to the test.

Fill in the Filename and the Variable Names, according to the number of columns you have. In this demo, we have two columns.

Go back to the sampler and change the parameters names from specific to general syntax. In this example, from London to ${toPort}.

Before running the test, don’t forget to update the thread group. To make sure you are going through all the rows, change the number of loops. Here, we have 5 rows in our CSV file, so we go through 5 loops.

Now, run the test to make sure it works.

Success! We can now see the different result combinations from the spreadsheet in the View Results Tree, just as we wanted.

Now let’s move on to assertions.

Assertions are important for ensuring that tests that are shown as successful, really are successful.

Why wouldn’t they be? Because even if JMeter recognizes the HTTP status codes as approved, sometimes the content of the page displayed is wrong. This can happen, for example, when the response code that was received was 200 OK, but we still receive a custom error due to a bad login credential.

Assertions help us manage this kind of problem - they ensure that the expected content is responded back to the user during the test.

To use assertions, first decide which data you expect to find in the response shown to your users. Going back to the website from our example, we can see our response message includes specific ports. That could be a problem, because we have 5 different combinations.

Assertions ensure that all the values we are examining (from the CSV file) are in the response message, meaning users are getting the proper response message.

Add an assertion:

In this case, we add a Response Assertion, but there are many different kinds: Size Assertion, comparing, looking into XML and JSON data, etc.

The Response Assertion can be applied to any of the response's components, like headers, response codes, the body, any non-encrypted documents it may include and even its sub-samples. Here we are adding a text response assertion.

To add an assertion, you can copy the value from your app and add it to the ‘Patterns to Test’ section. In this example, we change the port variables to reflect all of the combinations, so it says: Your flight from ${fromPort} to ${toPort} has been reserved.

Run the test. If the assertion fails, JMeter will show it as an error.

In the output, we can see the test was successful since the text responses show the different port combinations.

You could also see these successful results if you added an Assertion Listener.

Congratulations! You are now ready to use Data-Driven Testing and Assertions.

Stay tuned for part 3 - Timers