refcard cover
Refcard #246

Achieving Continuous Testing Excellence

In the world of automated testing, we need some well-defined guideposts to chart a manageable path to success. This Refcard aims to establish key benchmarks and best practices necessary to achieve them — creating a clear roadmap to continuous delivery with stability, predictability, and confidence.

Published: Jul. 29, 2019    |    Modified: Aug. 19, 2019
Free PDF for Easy Reference
refcard cover

Written By

author avatar Marcus Merrell
Director of Technical Services, Sauce Labs, inc
Section 1

The Metrics that Matter, the Benchmarks to Target, and the Best Practices to Implement

Think about your business for a moment. Maybe you’re a retailer hoping to capture a larger share of the eCommerce market. Maybe you’re a bank hoping to make it more convenient for your customers to transact online. Or maybe you’re a publisher looking to push new content your readers can consume on the go.

Whoever you are, whatever you sell, and whatever service you provide, you probably have a digital business, and nothing is more important to your brand or your bottom line than your ability to deliver a high-quality digital experience to customers. In the modern era, that means continuously delivering fast, visually appealing, and flawless web and mobile applications.

Most organizations have replaced the traditional waterfall approach to software delivery with modern agile development methodologies. They believe that doing so will accelerate release cycles, improve the quality of applications, and lead to a better overall experience for users.

By all accounts, we should be delivering software better and more efficiently than ever before. And yet, the opposite is happening. Despite our collective commitment to speed and agility, release velocity is actually stalling, with the percentage of organizations releasing software on an at least monthly basis declining from 36 percent in 2017 to 27 percent in 2018, according to Forrester.[1]

[1] Forrester: The Path To Autonomous Testing: Augment Human Testers First, Jan. 2019

Section 2

The Continuous Testing Roadblock

Why is release velocity stalling out? The answer is testing: specifically, continuous testing.

According to a recent Gitlab developer survey, testing remains the most common source of development delays, cited by more than half of respondents. Most development teams are unable to implement continuous testing effectively throughout the software development lifecycle. As a result, they wind up compromising on either release velocity or release quality (or both), undermining their most ardent plans to become more agile.

The good news, though, is that if you can overcome the continuous testing roadblock, you can deliver quality software with the stability, predictability, and confidence that Agile promises.

Even organizations that have shifted to agile development still tend to take a legacy approach to testing, throwing it over the wall to a dedicated QA team as an isolated (and usually final) step in the delivery cycle. This isolates the testing team from the Product Owners and Developers, and often leads to disparate tech stacks and impenetrable barriers between QA and the rest of the organization.

Within this isolated world, even the most sophisticated teams are still too reliant on manual testing. No matter how large your pool of human test resources, you cannot scale testing to keep up with the growth of your business and the expectations of your customers without automation.

Then, too, those organizations implementing test automation are often doing so poorly. Test automation is a complex endeavor and requires engineers who understand your domain, the test automation craft, and software development — a hard-to-find mixture of skills. Without these skills, and a well-defined strategy, either your release velocity or product quality will eventually suffer. Usually both.

This is a preview of the Achieving Continuous Testing Excellence Refcard. To read the entire Refcard, please download the PDF from the link above.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}