Measuring DevOps: Why Customer Experience Matters

Sometimes a DevOps success story is not exactly what it seems. Consider this likely scenario: Let’s say a company—we’ll call it Gamechanger Industries—has a fantastic idea for an application that will create a new, recurring revenue stream from the company’s existing customer base. A small project team of business, developers, operations, and security talent is assembled. Aligned with the business goals of generating recurring revenue, the team develops and deploys the new application at record speed. Customers download it in droves. It’s a DevOps success!

But wait, it turns out that after a few tries, customers stop using the app. In fact, many delete it from their devices. What happened?

One culprit could be a customer experience that didn’t meet expectations. In our latest ebook, “DevOps Without Measurement Is a Fail,” we take a closer look at what type of customer experience metrics we believe your DevOps teams should be tracking, in addition to what we think are the five critical drivers of DevOps success. To learn more about measuring the impact of your DevOps efforts, you can download the full ebook here.

It’s All About the Customer

In the case of Gamechanger Industries, was the app too confusing to customers? Was it too slow? Were there too many steps involved in critical interactions? Did the team dissolve after the release and move on to another project? If the customer is too frustrated to complete the transaction that generates additional revenue for the company, the DevOps approach has failed.

If this were your company, you wouldn’t know whether customer experience was the problem unless you were measuring and tracking the right KPIs within your software. These customer experience KPIs could include:

• Perceived response times of key transactions
• Frequency of key transactions
• Number of visits per user/per week
• User growth rates
• Funnel/conversion rates
• Amount of time spent in app
• A/B test results
• Customer satisfaction

The moral of this story is that aligning innovation and software development with business goals isn’t enough. You need to also link those aspects of success to key areas of the customer experience and measure how well you are doing.