Software Testing Takeaways From the 2018 Accelerate State of DevOps Report

The “2018 Accelerate State of DevOps” report is the brainchild of Dr. Nicole Forsgren, Gene Kim, and Jez Humble at DORA (DevOps Research and Assessment). Based on five years of research, with over 30,000 data points from thousands of companies, the project aims to understand precisely what practices enable teams to deliver better software faster.

Though it’s still hot off the press, the new report is already the talk of the DevOps community. We think it should become the talk of the software testing world as well. Testing and quality are discussed throughout the 78-page report—which now includes a section dedicated to Continuous Testing.

Here are 4 key takeaways for software testing professionals:

1. DevOps Matters to the Business — So It Should Matter to You

DevOps isn’t a fleeting trend. You can’t wait it out and hope it goes away. It’s delivering measurable business results, and adoption will only increase in the foreseeable future.

The research shows that elite performers and high performers (the report groups respondents into 4 categories: elite, high, medium, and low performers) are more likely to meet or exceed their business goals in terms of

• Profitability
• Productivity
• Market share
• Number of customers
• Quantity of products or services
• Operating efficiency
• Customer satisfaction
• Quality of products or services provided
• Achieving organization or mission goals

Even if you haven’t been personally impacted by DevOps yet, your organization probably has DevOps on its radar. If not, you should start preparing for DevOps anyway. Organizations that aren’t considering DevOps are likely to become disrupted rather than become digital disruptors. This means that you’ll probably be looking for new career opportunities in DevOps-driven organizations within the not-so-distant future.

2. Testing Must Change — Significantly — to Keep Pace With DevOps

Consider this:

• Elite performers deploy code on demand, multiple times per day.
• At elite performers, less than one hour elapses between code commit and production.

The rate of change is accelerating—fast. Testing must, somehow, keep pace. Evolving testing for DevOps is critical for realizing DevOps’ full potential in terms of accelerated delivery, agility, and all the derivative business outcomes outlined in the previous point.

3. High Rates of Test Automation (~90%) Are Essential for DevOps

At elite performers, manual testing represents only 10% of the total effort. Even the 30% manual testing rate for low performers is significantly better than the industry average of 80%+ manual testing (including teams that have not yet adopted DevOps).

The importance of test automation is underscored by the findings of a recent Forrester report that’s focused specifically on what quality practices separate DevOps leaders from DevOps laggards. That research found that DevOps leaders are:

• Transforming software testing into Continuous Testing by adopting five core practices, such as automating end-to-end functional testing and integrating testers into cross-functional teams.
• Almost twice as likely to consider automating the software quality process to be a “critical business differentiator” (53% vs. 27%).
• Significantly more likely (by 23 percentage points) to have high levels of automation for key testing and QA processes (test case design, functional test automation, test data management, etc.).

4. Quality Is Correlated With DevOps Success

Elite performers have a 7X lower change failure rate than low performers. They also spend 2X less time fixing field-reported defects. Exactly what DevOps practices are contributing to the impressive quality outcomes being achieved in the top-performing organizations? And how are different quality practices contributing to DevOps success? This is certainly an area that’s ripe for further research and analysis

Want to learn more about this report’s findings—and what it all means for software testing?

Join Nicole Forsgren, PhD, CEO and Chief Scientist at DORA and Wolfgang Platz, Founder and Chief Strategy Officer at Tricentis, for a webinar to explore Continuous Testing’s role in DevOps success. The focus will be on helping IT leaders understand:

• Why and how high-performing teams make Continuous Testing a linchpin of their DevOps process.
• What specific strategies and practices enable high-performing teams to deliver faster and dedicate more resources to innovation.
• Which quality-related technology and process investments have the greatest impact on DevOps success.