Black Friday 2017 will take place on November 24. This might seem a long time from now, but it’s never too early to start preparing for peak traffic and making sure your website doesn’t crash on the big day. In fact, some retailers start their Black Friday performance testing as early as February! But no matter when you start, just make sure you leave enough time for fixing bottlenecks, errors, and bugs due to high traffic spikes.
Load testing before Black Friday, or any big event, is important for companies. The cost of failure is direct loss of sales and money, a harmed reputation and customers moving on to competitors or stopping using the solution. But failure can be easily avoided, by load testing.
This blog post will cover best practices and tips for preparing your website or app for Black Friday, and is based on a webinar by our Developer Advocate, Dave Karow, which you can view from this link.
Creating an Effective Test Scenario
When creating your test scenario, build a test based in reality, not “happy paths.” “Happy paths” are easily predicted paths - a customer arrives at your homepage, clicks on search, puts items in a cart, checks out and provides credit card details. But the real world is messier, and you need to find out what your real test scenario patterns are and how many users to test. How can you do that?
Different tools can give you insights into your traffic patterns:
- Google analytics or Adobe analytics - show you how people behave. Look at traffic scenarios from other peak events, busy days and busy hours. This will give you an idea of what to expect, and therefore what to test.
- APM tools - show you what your infrastructure is doing when your peak hours arrive. Look for hotspots and the areas you know raise issues, and mark them as areas you need to further test.
- Logs - provide you with statistics about the rate of arrival, time users spend on a page, etc.
In addition, get information from your colleagues. Talk to product managers about future product plans. Learn from customer success what makes customers happy. Listen to what marketeers are doing to increase traffic to the website, to learn which pages are the most valuable.
After you have all this information and created your test scenarios, you can start building different kinds of tests.
Running and Scaling Your Tests
Preparing for a big event like Black Friday requires making sure you cover your system from end-to-end. So start small and build your way up, with these tests:
- Sandbox testing - a test that runs the test scenario for one user. The goal is to make sure the test does what it’s supposed to do. No point in piling up the load and trying to discover why it isn’t working, and then figuring out something was wrong with the test.
- Calibration testing - a test that ensures the system isn’t part of the bottleneck, e.g. making sure the system isn’t stressing for memory before upping the load.
- Performance testing - testing your load. We recommend you work your way up to the number of users you want to test, e.g. start with 10% of the target load and ramp up slowly. This gives you time to see your system’s symptoms when handling loads. If you’re in a rush to get to the top level, you won’t be able to identify issues along the way.
- Spike testing - a test for getting ready for a sudden spike of users. Useful when you have a site that goes live at a certain time, or for preparation for Black Friday peaks. After the strain, scale back the load to see how the system recovers after the spike.
- Soak testing - a test that strains your system over time. Black Friday is a long day, so make sure you can run straight for a long time. Certain things will appear when running for a long time, like memory leaks or queues that unload slowly - know what they are so you can fix them ahead of time.
- Capacity planning - a test that deliberately takes your system to failure, so you know where things break. This gives you a better understanding of what needs to be solved and how to solve it.
5 Tips for Smart Load Testing
1. Construct Realistic Funnels
The number of people on your landing page isn’t the number of people checking out. This could be due to your system’s capacity, and you should check that. So check your website or app to see where people drop off, and then test accordingly.
In this screenshot, you can see that we configured that 90% of users should go from home page to search, 60% to put an item in their cart, and 35% to check out. Now run the test to see if the app can handle the load.
2. Toggle 3rd Parties and Features Set on the Fly With Properties
When running a test, different system parts might get overwhelmed by the load. You can shut down the test, but you can also shut down parts of it, to keep the test running and to keep examining other system parts. You can also use this approach to shut down 3rd parties, since you probably want to load test your own system.
One way to do that is with the JMeter If Controller, if the test is not too complex. The elements under the If Controller are executed only if the condition is true. In this example, the 3rd party features are executed only if the Control 3rd party is “yes”, meaning it works properly.
3. Decorate Headers so You Can Monitor the System on APM
Add a header to all the requests that are going out. The headers will enable you to analyze load testing results by identifiers like timer name or script name, check which tiers have problems, examine your DB for a bottleneck and ensure you JVM and host are healthy.
On JMeter, you can add headers with the HTTP Header Manager and then see them in the different listeners. On BlazeMeter, configure headers through APM integrations and then analyze results there.
4. Adjust Your Requests Per Second In-flight to Check if the System Can Recover
When running a test, it might start to show issues and problems during test run. Instead of stopping the test, adjust the RPS during the test run, to see if your system can recover.
In JMeter, you can use the Constant Throughput Timer. Adjust the RPS as a value that is defined in the command line, and then run the test. If you need to change the RPS, change the value in the command line. In BlazeMeter, you can change the RPS value in the JMeter “Remote Control.”
5. If You Use a CDN, Test It
CDN testing is crucial for ensuring your system is fast and has low latency. Therefore, you need to load test it, especially before big events. CDN load testing requires testing from different locations, which you can run through JMeter if you set up different machines. Or, you can use BlazeMeter.
Get ready for Black Friday 2017. Start load testing now and ensure your site or app can handle high traffic this Black Friday.