I heard about Gatling long back, but recently got the chance to work on it for one of the project. It's a great tool, key features I liked is the recorder, simple to understand scripts (SCALA), easy to configure and learn. One can get started with the tool within 2-3 hours, though building expertise will take more time. Gatling team has provided enough documentation to start using it.
I am not going to describe the basic configuration and quick start guide, that is already available on Gatling site. While testing I had few observations/thoughts/learning so thought would be beneficial for others as well.
Gatling is a load testing tool. hmm “load testing”, What I understand from it is, this tool will help you to put load on your server and measure the throughput in different scenarios, this will also help you to benchmark the load your server can take. Though tool can be used for stress testing as well.
Thoughts on How One Can Build a Strategy to Perform a Load Test?
Get answers for below questions from your stakeholder, and based on that one can build really good strategy:
- We need data on how many users we are expecting to use the application, at a time and gradually how many users will be online?
- What will be the peak time load?
- How many simultaneous users will login?
- How many user will keep the session open?
- Which functionality on web page is heavy from resources perspective?
- What should be the ramp-up time?
- What is expected response time?
- What is the average network bandwidth users will use?
- Which functionality will be used most, do we need to test it differently? e.g. Search
This step is crucial as it will help you to setup right scripts/scenarios.
Am I Putting Load on Appropriately or Am I Measuring the Request/Response Time Appropriately?
I really wanted to cover this part, if you really want to measure the (near to) realistic response of your application then you need to consider (There might few more which I am not thinking nowJ) how your users going to use your application. Response time of any request is depend on multiple factors, server configuration, underlying resources, caching implementation consumer device/machine configuration and network speed. If we miss considering this aspects we may end-up getting wrong impression of the system.
If one has 20 MBPS internet connection, when he make a request in browser he may not get the 100% bandwidth, there is some bandwidth reserved for OS, there will some applications running in background. So we need to find what the average internet speed users will use is.
If it’s a mobile site then it will be on lower side, if its mix then we may need to have mix bandwidth.
If your setup is on single machine and you are running load test for 500 users and consider the request size is 200 kb, so your script will be consuming 500x200 = 100,000 kb (100 MB). So, in other words, we are expecting 100MB to be downloaded in certain time. Does it look problematic to you?
So if you are running Gatling scripts from Jenkins which is used by other applications then you really need a strong strategy for load test execution. My recommendation is to use Jenkins to trigger the Load test on separate machine or machines.
What If I Don't Know How My Users Will Use My Application?
In initial release we may not have complete picture, one can take inputs from stakeholder and can build strategy accordingly, but we will also add a monitoring so that when next time we want to run the load test we will have good inputs to load strategy.
How Should My Load Test Setup Will Be?
I will suggest to take it to the cloudJ. To make this article simple let’s consider I got configured 20 machines (I used OpenStack based solution), Created an image with Gatling and Java 8, triggered the Gatling remotely on all machines, generated only log files, copied those log files to one machine (will call it as master) and generate the HTML report from those log files. We can use the network throttling tools to limit the bandwidth of each connection. I limited it to 2 MBPS whereas the machine had connection of 40 MBPS. On each machine I ran test for 50 users. One can change the number of machines based on requirement, only thing is we need consider is to make the simulation close to actual user scenarios.
Why Should I Limit the Speed of Internet?
This configuration or consideration will play a major role if your requests are taking more than 10 seconds, when tested with the bandwidth actual user will have. If a request is taking 10 seconds of time that means it's holding the connection to the server for 10 seconds. By this, we are trying to simulate near real-time load on server. In other ways, your server may be capable of serving 1000 user with high-speed internet but it may fail if the internet speed is slow.
To demonstrate it. I tested one application which is built based on single page architecture and can be accessed by mobile devices and the desktop machines. I ran a 200 user test, with a 1-minute timeout and 20 seconds ramp up time (in 20 seconds 400 user will hit the URL, randomly), my internet speed roughly was 2 MBPS for each connection. 500 requests failed out of 4456 due to timeout (1 minutes).
I ran the same test with 4 MBPS limit to each request and none of the request failed. But we observed a spike on CPU usage.
My Takeaway From the Test Result
If number of users are who use average internet speed like 2MBPS then adding few more servers with low configuration will help for availability. (Horizontal scaling)
Whereas if users are going to use high-speed internet, I may need to add more CPU capacity. (verticle scaling)
"Great Power Comes With Great Responsibility"
If you as a developer would like ti use Gatling in Jenkins and to run the Gatling daily/weekly then we need consider the resources your application do use, it might happen that the DB server your application is shared with other application and due to the load tests other applications are down.
Your application may be configured to handle a high load but does the dependent applications are also configured for that load?
So if the application runs on its servers and not depend on other applications or doesn't shares the DB servers with other application then you have more freedom. If not then you need to better plan on how/when you will run the load test.
Hope this help you to define right strategy and right setup of your Gatling or any other tool you want to use for load testing.
Be on the lookout for Part II: How to Configure Gatling Using Cloud?