Load Testing With KPIs Part I: What Are KPIs?

KPIs, or Key Performance Indicators, are metrics that enable measuring our results and success, according to parameters we choose to be relevant and important. Organizations use KPIs to evaluate themselves and their activities.

In the context of load testing, KPIs show user and traffic measurements for websites and applications, in order to best determine if they can withstand a certain amount of load to its backend servers. 

Since we don’t want our website or app to crash during Black Friday, the rush for Beyonce tickets, or after we’ve updated new features, here are the seven KPIs you need to know when you load test with JMeter:

1. Number of Users

To make sure websites can handle heavy loads created by the usage of many people, we simulate concurrent active users through virtual users. These virtual users act like “real” users on the website. By simulating real users, developers, DevOps and QA engineers can discover bottlenecks that could occur during real-time daily use or traffic spikes.

2. Hits Per Second

Simulating the number of users can give us a good idea of the load, but it’s not by itself an accurate enough measurement. We also want to know how many requests are generated to the target server from users’ actions. Hits per second is the average number of samples initiated per second. By understanding the correlation between the number of users and hits per second, we can accurately simulate and measure the types and loads of usage of the website, thus validating that the site’s performance abilities are adequate. 

3. Errors Per Second

After measuring the number of hits per second, JMeter identifies the number of errors per second. A high errors-per-second rate can indicate a bottleneck that needs fixing before going live. 

4. Response Time

We’ve measured users and their actions, now it’s time to measure how long it takes the system to process a request. This parameter measures the amount of time it takes from the first byte of data to leave the user until the last byte is received by the user. Response time shows how the target site is performing from the user point of view, meaning how long it takes to receive the text and images requested by user actions. A site might be able to handle 10,000 concurrent users viewing its homepage, but if it takes too long to process their request for tickets, their votes or their payment methods, it is not handling the load and customers may decide to leave.

5. Latency

The measurement of how long it takes from just before sending the request to just after receiving the first response. Part of the response time KPI, latency helps us to measure and understand the inherent network delay during transmission of data from client to server. 

6. Connect Time

The measurement of how long it takes the user to connect to the server and the server to respond, including SSL handshake. It is a part of the response time KPI, but it is important for isolating SSL performance as a bottleneck.

7. Bytes/s (Throughput)

The measurement of the average bandwidth consumption that is generated by the test per second. This measures the amount of data flowing to and from the servers. Isolating this KPI is important for ensuring your network interface controllers are performing properly.

Congratulations! You now know which KPIs you need to measure when using JMeter.

Stay tuned for Part II, explaining which KPIs are recommended to be correlated with each other.