5 Tips for Scaling to One Million Users

Load testing is a way to ensure your website or app doesn’t crash under heavy loads. When load testing, one of the basic things you need to do is scale the number of virtual users you are testing. Depending on your business needs, you might need to scale to huge numbers, maybe even to 1,000,000 users.

Here are five tips that could be helpful for performance testing scaling.

1. Plan in Advance

 In many cases, a huge surge in the number of users can be anticipated. For example, traffic spikes are expected on the day of the big game for food delivery places or live streaming TV sites, on Black Friday for retail stores, on the day of opening ticket purchase options for popular music festivals or movies, on first weeks of releasing a new app or website, and more.

Therefore, we recommend you start load testing for larger numbers at least a few months in advance. This ensures you have enough time to run the tests properly, find your breaking points, fix whatever bottlenecks need fixing, and run the tests again to make sure everything works. The number of users you should be testing depends on your business needs and your expectations.

Anyway, we recommend you incorporate load testing into your Continuous Integration processes. This diminishes the chances of unexpected traffic patterns catching you by surprise and crashing your website.

2. Record Your Test Scenario

The easiest and most useful way to build your test is by recording your user scenario. While JMeter offers recording options, the most useful and easy-to-use recorder is the BlazeMeter Chrome Extension, which is free.

To record your load test scenario:

• Add the recorder to your Chrome from the Chrome store.

• Type in a test name.

• Click the Record button and go through the scenario you want to test.

• Click the Stop button.

• Edit if necessary.

• Export to .jmx and upload to JMeter, or click Play to open it in CA BlazeMeter.

Recording your test scenario is recommended for any load test. However, since scaling to high numbers requires some time, you need to simplify as many processes as you can. The recorder does just that. 

3. Remove All Unnecessary Data From Your Test Script

All load tests take up a lot of resources, but massive scaling takes up even more. Make sure your resources don’t crash due to bottlenecks unrelated to what you are trying to test. To do that, remove any unnecessary data.

For example, we assume you won’t be running massive scaled tests on JMeter (which probably supports a few hundreds to 1K users on a single machine), so remove the listener (but be sure to keep it when running the test on JMeter).

We also recommend you remove all embedded resources. This can be done by unclicking the Retrieve all Embedded Resources box at the bottom of each HTTP sampler:

4. Adjust the Test Gradually 

Starting your first test with the high number of users you want to test might work, but it’s also very probable that your system will crash. The reason could be either the test itself wasn’t constructed properly or that your system broke due to the high number of users.

In each case, you want to find out when and why the system broke. Therefore, we recommend you test gradually and adjust the number of virtual users moderately.

• Start out by running the test locally on JMeter for one thread (virtual user) to make sure the test works.

• Go to the Thread Group and manually change the number of users to a few dozen or a few hundred. If the test works, keep adding users.

• After reaching a certain number of users, your local machine might not be able to handle the loads. If you still need to go up, you can use tools like CA BlazeMeter (a type of JMeter in the cloud) to continue scaling.

• Run the test in SandBox mode to make sure it works.

• Gradually scale to the number of users you need. After reaching the largest number available for a single console, go to CA BlazeMeter’s multi-test option and duplicate the tests until you reach the number of users you need. Read more here.

By gradually adding users, you can determine your breaking point and identify bottlenecks, like high memory usage or improper DB usage. Fix the bottlenecks and keep testing until you reach the breaking point that is higher than the number of users you decided on handling.

5. Constantly Analyze Test Results and Engine Health 

When load testing, two of the most important health parameters we look at are CPU and memory usages of our website. You need to determine the usage levels that you don’t want your system to cross, but if you reach 75% CPU and 85% memory, you’re approaching a red line that you don’t want to cross.

To monitor your system’s health, look at the reports the load testing systems offer and analyze different KPIs and the correlations between them. CA BlazeMeter offers an Engine Health report, but you can also analyze these parameters on Taurus, an open-source test automation tool.

It’s important to analyze the results in real-time after every test you run and over time. This ensures that you get a full picture of your system’s abilities, weaknesses, and bottlenecks. They also let you see what happens to your system when changing it, like adding features or changing the system’s architecture. Finally, they let you see if the fixes you’ve made are really helping.

Ensuring your system handles heavy loads is important for keeping users and customers coming. We hope these tips will help you when scaling to high numbers.