The Pulse of QA: How Healthy are QA Organizations in 2019?

Product quality is more important to the success of businesses than ever before. In a software market flooded with an ever-increasing pool of options, businesses, and consumers alike demand high-quality, delightful experiences from the platforms and applications they use daily.

In this market, getting QA right is essential. For the 2019 QA Health Survey, we polled over 250 software professionals to get a sense of how healthy quality organizations are right now. Read on to see some of our key findings, and to download the full results of the survey.

Manual Testing is Still Essential for Strategy-Driven QA Teams

Automation has been a major push in the QA space for years, but 89% of all teams have at least one full-time manual QA tester. While many teams shared they want to automate as much as possible, many have found that total automation is too large and complex of a task to tackle wholesale, and are taking a more nuanced approach to how they think about test execution methods.

“Currently, we’re trying to automate high-risk areas across the platform,” responded one QA Manager. “Originally we were looking to automate individual areas thoroughly, but we were missing necessary coverage across the board, so we switched strategies.”

As teams continue to look for ways to test faster and more efficiently, they’ll need to find ways to balance the range of testing methodologies and tools available to create the right mix of speed, accuracy, and cost.

Test Coverage is an Ongoing Struggle

Only 20% of teams feel they have a high level of test coverage, and another 31% say that they hit coverage goals “almost every time.” But this means that nearly half of QA teams still struggle to achieve a level of coverage for deployment cycles that gives them the confidence they want.

Coverage has always been a challenging area for QA teams — how much is too much? How much is not enough? Many teams still take a “more is more” approach and try to cram as much testing as possible into every launch. But this approach often perpetuates the “boom and bust” cycle of QA that only becomes more ineffective over time. Teams that want to balance test coverage with speed need to think smarter about how they build (and maintain) that coverage.

Developer-Driven QA Has Unique Challenges (and Benefits!)

In organizations where QA is part of the development organization, QA metrics are more likely to be measured regularly; 66% of dev-driven QA teams have regular QA metrics, compared to only 42% of teams with a separate QA organization. Given how critical the right QA metrics are to product quality, this is a great benefit for these teams.

However, development teams tend to be less strategic about their approach to QA testing. Only 49% of dev organizations said that they have a QA strategy (compared to 59% of QA organizations), and only 41% stated that their QA goals are aligned with business goals (compared to 55% of QA organizations).

As teams move faster than ever before, moving QA under the umbrella of development can be a great way to keep the team streamlined and well-aligned. But healthy product quality requires these teams to take a step back and think strategically about how QA impacts their business goals.

Download 2019 QA Health Survey Results

Want to learn more about how your team stacks up against other QA teams in the industry? Download “Taking the Pulse of QA” for the results from our 2019 QA Health Survey to see what else QA leaders had to say about the state of quality for their organizations.