DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Deployment
  4. How to Build and Execute Performance Tests Before Black Friday and Cyber Monday

How to Build and Execute Performance Tests Before Black Friday and Cyber Monday

Have you already started load testing for Black Friday? If not, don’t worry. There is still enough time to discover bottlenecks and fix them.

Noga Cohen user avatar by
Noga Cohen
·
Oct. 21, 16 · Opinion
Like (2)
Save
Tweet
Share
5.01K Views

Join the DZone community and get the full member experience.

Join For Free

Have you already started load testing for Black Friday? If not, don’t worry. There is still enough time to discover bottlenecks and fix them.

But don’t wait any longer - testing your site or app is important to make sure you don’t lose money, send customers to competitors or create a bad customer experience. If your site performs well on the big day, you may also gain customers from your competitors, if their own sites fail.

Here we’ve outlined how to prepare your website or app for Black Friday - by building and executing performance tests. This blog post is based on the special Black Friday webinar we held, which you can check out here.

1. Create and record user scripts according to your application’s business process

Identify the user flow your customers perform during the event. Use Google Analytics, APM tools such as New Relic, server and database logs such as logentries and logz.io, and last year’s data. We recommend you also use your team: Customer Success for determining flows for your most valuable customers, Product Managers for testing new features, and Marketeers for learning what kind of new traffic is expected to come.

After you determine the user script, start recording.

Let’s look at this example.

- Open the BlazeMeter Chrome Extension, available at the Chrome store

- Change the name of the recording and press record

recording user scripts for load testing

- Execute your user script

executing user scripts during recording

- When you’re done, open the BlazeMeter Chrome Recorder again and end the recording

- By clicking on the pencil icon on the recorder, the editor opens up. In the editor, you can see the different values from your recording. These can be edited.

edit load test recording

- Export the recording to JMX (the button is on the top right)

export recording to jmx

2. Configure your performance tests and run them

You can open the test in JMeter and in BlazeMeter.

This dashboard, in BlazeMeter, demonstrates a test run for:

- 2,000 virtual users (VU)

- 5 engines

- 400 threads

- 1,200 seconds ramp-up duration (we recommend the ramp-up for your first performance test is 30 minutes - this ensures you reach your maximum value and can accurately identify issues)

- 30 minutes duration

- and from Iowa

In addition, CloudWatch is used for monitoring and New Relic is for APM.

configure load test on blazemeter

Additional configurations include:

- Executing 3rd party branches (to shut the branch down, go to the Remote Control and change the value to “no”. This takes less than a minute).

configure 3rd party branches on load test

By setting Test Failure Criteria, you can easily receive feedback from the product into the performance test.

configure test failure criteria on load test

Here:

- If the overall Average Response Time is more than 5 seconds - the application is too slow

- If the overall Error Percentage is more than 2 - there are too many errors

- If the Response Time specifically for booking a flight is more than 10 seconds - booking a flight takes too much time

- After you finish configuring the test, click save, and start the test

3. View and analyze your results and identify your bottlenecks

The BlazeMeter dashboard presents multiple reports that let us identify the issues.

This Summary Report displays a summary of the test. It is clear the application failed, as the Error Percentage is more than 25%.

performance test summary report on blazemeter

We can also see the test is saturated somewhere between 700 and 800 users, since the hits per second (Hits/s) stopped increasing even though the number of VU kept increasing.

users, hits/s and errors analysis

By looking at the 2 graphs side by side, we can see how the Response Time bottleneck turned into errors.

report analysis shows response time bottleneck

For a more in-depth view, go to the Timeline Report. You can play around with the parameters you want to see. Here, we added the Average Call Time and the Throughput through the New Relic APM.

timeline report on blazemeter

When the errors started increasing, the throughput decreased significantly. This means that some requests didn’t reach the application since the Firewall or Load Balancer failed.

If you go to CloudWatch, you can see that the surge queue length fluctuates. This means that the Load Balancer can’t fill the requests from the load test.

cloudwatch report on blazemeter

What should you do? Either call Amazon support to ensure they keep the Load Balancer warmed up on your big day, warm it up yourself, or change your load balancer.

Now let’s look at the Failure Criteria we configured before:

failure criteria load test analysis on blazemeter

While the Response Time was satisfactory, the Errors Percentage was too high and the RT for “Book a Flight” was too high.

Test Configuration Tips

- Stress Test - Black Friday traffic peaks at 10:00 AM. Make sure your app can reach the maximum value you determined in a short period.

- Soak Test - The US has different time zones and customers can still be scattered throughout the day. Test the maximum value for a longer period - preferably 6-12 hours.

- Reach your limit - Even if you can handle the load you set for yourself, discover your bottlenecks. Add VUs and test for longer durations. You might be surprised during Black Friday.

After Black Friday

We recommend you run load test every 2-4 weeks. As you can see, the process is easy. By adopting a Continuous Delivery (CD) method, you can discover bottlenecks more efficiently and ensure you are always on point with your business plan.

Multiple tests over time also enable you to compare trends and changes, and analyze how you perform in real-time events.

Taurus, an open-source test-automation tool, integrates load testing tools into Continuous Integration tools like Jenkins.

Testing Continuous Integration/Deployment Load balancing (computing)

Published at DZone with permission of Noga Cohen, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • A Comprehensive Selenium Locators Cheat Sheet
  • Deduplication and Data Stewardship Process in MDM
  • Key Considerations When Implementing Virtual Kubernetes Clusters
  • Pipes And Filters Pattern

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: