Functional testing and non-functional testing of an application are slightly different from each other. In terms of ISTQB, functional testing has structured tests scenarios. But when it comes to non-functional ones, we have performance testing and usability testing where the user’s perspective comes into play. Then, we have uncertainty as stated below:

QA Engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 999999999 beers. Orders a lizard. Orders -1 beer. Orders a sfdeljknesv. But the first customer comes in and asks where the toilet is? The bar bursts into flames and the customer kills everyone.

We all know that customers are different; they don’t do what we expect them to do in our mobile app or web apps. That’s why as Loadium service team we prefer to use 20 percent of manual testing scenarios and the rest are formed with complex scenarios gathered from the product, marketing, and business teams. This is because they know what the user is doing in the application.

No user enters a website, adds a product directly into a cart, and makes the payment. This is the scenario that we all want, but this is not the case, unfortunately.

So, How to Create a Realistic Load Test?

1. Geography

User’s geographic location plays a significant role when we do performance testing. Their distance to the data center, the number of hopes that requests do between the server, and the client plays an important role. During that transmission process, data will pass through locations with different backbone speeds. Therefore, distributing the load into different geolocations would be a good start for your performance tests.

2. Devices and Browsers

What difference can a device or browser make to a load test? Aren’t we simulating HTTP requests? Yes, we are, but a user’s think time depends mostly on a browser because of rendering time. Not every browser renders the pages at the same speed and the user takes an action on that page. A review of a browser’s rendering can lead to identifying how long the user must wait for the individual steps to become active.

3. User Behavior

No user buys the same shoes every time in your website. Nowadays, every web or mobile app is using a strong caching mechanism for performance issues. So, users mostly get the response from cache servers without actually going to the actual server. Then, randomize your test data. Don’t navigate to the second page on a search page. Navigate to the 10th page. Maybe your caching servers cache the first 20 results, then go the 21st. Force your server to process data during performance tests. If there’s no parameterization or randomization, we cannot talk about a good performance test.

Google Analytics can provide a good view on parameter variability as they are done by actual users. So you should be close friends with marketing and product teams.

4. User Paths

As we stated earlier, not every user behaves the same. Let’s think about a user who buys a movie.

Scenario 1: User searches for the movie via the search bar. Selects movie, adds it into the cart. Then he/she pays without being a member.  

Scenario 2: User logins to the website.Searches for the movie. Adds it to cart and buys it.

Scenario 3: User browses the categories. Finds the actor. Finds the movie. Adds it to the cart and leaves it like that.

Most of the scenario, the user reaches the same destination but each user performs different steps. So, you need to analyze every scenario according to its impact on application architecture. You need to find the most extensive CPU consuming scenario but maybe perform it with a small user group. Those correlations will define the quality of your performance test.

5. Network Behavior

Each user has different network speed each time the app is used. Think about someone on a train, they are using a mobile app with 5G speed, and then, it becomes EDGE connection in a tunnel. Therefore, the network should be taken into account seriously. You need to understand what happens to the user who has slow connections.

All those things are key factors of a performance test. Loadium offers you to generate load in different geolocations with different bandwidth types. All the rest (users path, user behavior) is up to your JMeter skills.

Happy load testing!