If you’ve been here for the past few years, it would have been hard for you to miss the digital stampede from ticket-based processes to continuous delivery. But somehow, this transition has skipped over load-testing processes. This is probably because performance problems are hard to fix, as they are removed from the code.
But many open source tools enable developers to easily “shift left” and run load tests as part of the continuous delivery process. With these accessible test performance abilities, you can start tests earlier in the process and verify each step along the way.
This blog post will go over running load tests on Taurus (an open source test-automation tool), Gatling, and Locust. We will show how to run tests both locally and through the cloud by using Taurus and BlazeMeter.
First, we will simulate a test, which we will use for each OS tool. This is the Simple Travel Agency, which we often use at BlazeMeter for demos:
Performance Testing with Taurus
Taurus is an open source framework for test-automation. This free tool runs performance tests for scripts from other open source tools, including JMeter, Gatling, Locust and Selenium. By extending their abilities and covers up complexities, Taurus provides a simple way to create, run and analyze load tests. Taurus uses YAML or JSON, which are much easier to manage.
This is the Taurus file for travel agency demo. As you can see, Simple is quite simple:
● The concurrency is the number of users making requests for the scenario—we are using 20 for now because we are running locally.
● The ramp-up time is the time it takes for users to reach the desired concurrency. This is the way to stagger the load and stimulate growing.
● Hold-for is the duration of the scenario.
● The scenario is the test we are running. It is in YAML file.
This is the test Taurus provides and keeps updating during the test run. On the dashboard we can see the hits, failures, response time and more:
To make the test run from different geographic locations, we use BlazeMeter, by adding the BlazeMeter personal token, changing the concurrency and adding locations:
The test is opened through BlazeMeter:
These are the results (for 20 users):
BlazeMeter enables developers to see how their code performs from different geographic locations. Stats are collaborative because they are viewed on the web. This means you can share them with your team. You can also zoom into different metrics and choose which ones you want to analyze—enabling you to figure out bottlenecks.
Performance Testing with Gatling
Gatling is a Scala based load testing framework. It works well with Maven and offers good reporting.
This is the same test in Gatling. The characteristics are defined in Scala. Changes are made both in the Taurus file and in the Scala file.
We now run the test through Taurus:
And these are the results on BlazeMeter for 1,000 users from two geographical locations, in the cloud:
As before, these results are collaborative and can be used to identify bottlenecks.
Performance Testing With Locust
Locust is python-based, very well designed for load testing and is easy to learn.
Here is a Taurus test using the Locust executor. This test uses two different concurrency levels, one for local, and one for cloud. The concurrency choice can be made from within the test by choosing the provisioning. This possibility is available for every run on Taurus.
The local results:
And in the cloud, for four locations: