Everything You Need to Know About Performance Testing

We love following the Ministry of Testing #30DaysofTesting challenges. For May's 30 Days of Accessibility Testing, we participated by talking about how top tech companies are integrating accessibility into their services, platforms, and routines. For July's 30 Days of Performance Testing, we wanted to discuss the fundamentals of performance testing.

Hopefully, by going over the issues, approaches, and outcomes of performance testing, you'll be able to check more than one box off your list for the challenge.

What Is Performance Testing?

Performance testing is used to see how well a software can handle user traffic. By putting a simulated demand on an application or website, it's possible to analyze breaking points and evaluate expected behavior. Specifically, performance testing is used to measure responsiveness, stability, scalability, reliability, and speed.

As a code change from a team that is continuously integrating new features and bug fixes can impact how an application looks and functions on different browsers and devices, it can also affect how quickly that application loads across machines.

This is why performance testing is so crucial to a well-rounded QA strategy - checking an application's performance and ensuring consumers are experiencing acceptable load time and site speed is foundational to high-quality software.

Performance testing also helps teams analyze activity so they predict traffic trends. This allows them to better prepare for breakpoints and site dropouts for the future.

Load Testing vs. Stress Testing

When we think of performance testing, we can usually separate methods into two categories - load testing and stress testing.

Load testing looks at normal conditions of your application - how quickly does it load when one user visits your website? How quickly will it load when 15 people are on your website at once? Load testing is used to make sure that software works as expected in ordinary circumstances.

Alternatively, we have stress testing. Stress testing challenges the limits of your application to see how much traffic it takes to slow down performance or break completely. By simulating 10,000 users, for example, a website will likely perform differently under that pressure than it will in everyday use. While stress testing encompasses unlikely test environments and user scenarios, it's important for analyzing risk and breaking points.

Additionally, many B2C companies and online e-commerce sites depend on stress testing before events where high traffic is expected such as Black Friday, the Super Bowl, and Election Day. If you aren't stress testing in these situations, just a few minutes of downtime could result in thousands of dollars in missed sales at a time that is critical to the business's success.

Monitoring

While performance monitoring is has a similar goal to performance testing - to ensure the that an application is functioning under expected traffic conditions - it's executed in an entirely different way. Rather than simulating user behavior, a performance monitoring tool like AlertSite is set up to watch the application, measure response time, and send alerts if the site goes down.

This is important for the same reasons that testing is important. While you want to be able to test the performance of your web application before your users encounter a problem, you also want to know if and when your users encounter a problem so you can promptly fix the issue before downtime noticeably affects your reputation or revenue.

What Happens When You Neglect Performance Testing

While some organizations are slower to adopt a complete performance testing strategy, they'll quickly realize that having no strategy will negatively impact bottom line goals.

Here are a few of the ways that performance testing affects business ROI, according to Radware:

• 51% of online shoppers in the US say that site slowness is the top reason they'd abandon a purchase.
• Shoppers remember online wait times as being 35% longer than they actually are.
• A 2-second delay in load time during a transaction results in abandonment rates of up to 87%.
• The total cost of abandoned shopping carts for online retailers has been estimated at more than $18 billion per year.
• 64% of smartphone users expect pages to load in less than 4 seconds.
• When faced with a negative mobile shopping experience, 43% of consumers will go to a competitor's site next.

As we witness a rise in mobile device use and multi-screen behavior, regular app updates, and demand from users for instant gratification with technology, it's evident consumers are less likely to use applications that don't meet their high expectations and more likely to leave your site and buy from faster competitors.

In turn, through regular load and stress testing, users will be more likely to stay on your site from the first time they land on the homepage through checkout. Additionally, by establishing trust as a fully-functioning application, they'll be more likely to come back.

Performance Testing Tools

Performance testing is not complete without the assistance of a few tools, whether those be open source, paid, or a combination of both.

These are the tools that teams find are integral for load testing, stress testing, and web application monitoring.

• JMeter - Apache JMeter is a go-to open source tool for load testing and measuring performance. As a purely Java-based application, it's used for recording, building, monitoring, and debugging on different applications, servers, and networks. People prefer JMeter because of its user-friendly installation and GUI, multi-threading framework, and visual results.
• Gatling - Gatling is another open source tool for performance testing on web applications and based on Scala, Akka, and Netty. It's used to help predict and detect issues with features that allow you to automate tests, record tests, edit scenarios, analyze bottlenecks, and share results.
• Fiddler - Fiddler is a free debugging proxy tool used to monitor traffic from web applications. It can be used on any browser, system, or platform and lets you see the total page weight, isolate bottlenecks, record traffic, and troubleshoot.
• Selenium - Almost all things testing are possible through Selenium, and while it's not particularly made for load testing, Dave Haeffner gives a pretty good rundown on how to use the beloved open source tool to make it work.
• WebLoad - WebLoad also has both a free and a paid version. Similar to Neoload, the free version offers 50 virtual users. Additionally, the Load Generation Console simulates large user loads on a local cloud, the analytics dashboard provides shareable report templates, and it integrates with other open source software like Selenium and Jenkins.
• LoadComplete - LoadComplete is used as a desktop tool for load and stress testing websites without requiring advanced programming or automation skills. It includes features such as record and playback as well as visual programming and allows you to generate load from VMs, local computers, and the cloud for a comprehensive performance testing and monitoring strategy.

Conclusion

As you continue the 30 Days of Performance Testing challenge with books, blogs, and podcasts, keep in mind the fundamentals of performance testing and the ways in which you need to leverage certain tools to apply specifically to your own web applications.

Keep in mind the high risk of not monitoring your software, load testing, or stress testing, and make sure your QA team has a strategy in place. By evaluating user behaviors and workflows, you can prepare for real-world scenarios without sacrificing scalability or customer loyalty.