2015 State of DevOps: Speed Meets Quality

The 2015 State of DevOps Report (from Puppet Labs in partnership with Gene Kim’s IT Revolution) is now out, marking the fourth year of the report covering how DevOps connects IT performance with business value. The report continues the previous years’ trend of comparing data from high-performing companies that have adopted DevOps against low-performing firms that have not yet embraced DevOps. This year, the report highlights how the performance gap between high and low performing companies has grown significantly: the DevOps movement continues to yield results. Here’s what we gleaned from the report:

High-performing companies are still deploying 30 times more often and 200 times faster than low performers—about the same as in 2014—but they are now recovering from failures 168 times faster and implementing changes 60 times more often. That’s a huge increase from 48 times faster recoveries and 3 times faster change implementations in 2014. There is a very important takeaway here: high-performing companies are getting better at what they are doing because they are doing it more often.

Change as a Competitive Weapon

Put simply, high-performing companies are still moving fast, but they are increasingly able to make changes more confidently and reliably. This speaks to the real value of DevOps, which is to be able to both move fast and not break things. Companies that can implement change reliably have a powerful weapon—they can experiment faster and ultimately out-innovate their competition. Experimentation and failure go hand in hand, so the ability to fail but recover quickly and learn from the experience is a competitive advantage that high-performing companies can leverage. It’s a weapon that helps them figure out what resonates with their customers and implement solutions before their competitors do.

The notions of agility and quality can seem to be in opposition, so it is empowering to see quantitative data demonstrating that both can be accomplished simultaneously. Comments in the report that “fewer things break because issues have been resolved earlier in the software development process” tangibly show the benefits of DevOps, with a nod to its kaizen manufacturing roots.

Quality can easily be overlooked in the service of agility, particularly at low-performing organizations just starting to experiment with DevOps. Seeing quality “shift to the left”—looking at quality holistically and baking it into the core development and deployment process—should encourage more risk-adverse companies to adopt DevOps. The learning process begins with figuring out how to change more often, and then managing to do so with less operational impact. That is what the report indicates is the trend among high-performing companies, and this combination of agility with quality is one of the necessary ingredients for innovation.

No “Pain,” No Gain?

The report also uses a clever approach to quantify both the ability and willingness to change, what it calls Deployment Pain. The report challenges companies to think about “How painful are deployments?” and suggests this is a good indicator for how well you’re doing with DevOps adoption. High-performing DevOps teams aren’t scared of deploying to production—this year’s stats indicate that they are doing so more often, and they’re doing it with fewer failures and quicker recovery times.

Fear related to deployments is only natural. Production is where all the validation happens—validation of the infrastructure, validation of the architecture, and validation of software deployed. If things go bad with a deploy, the dynamics change to reactive firefighting, which can greatly disrupt the flow of work. But the State of DevOps report highlights how deployment risk can be mitigated by making smaller and more frequent deployments and measuring the impact of each deploy. (Note: We’ve blogged about the importance of measuring deploys before, because we agree about the importance of measuring the impact of change.)

DevOps Fights Burnout!

As a recovering old-school ops guy, it was nice to see the report specifically call out the the issue of burnout, which is closely tied to change. Fighting burnout is an endless battle for many individuals in operational roles. Ops has long been regarded as a thankless job and people in these roles become accustomed to rarely receiving praise. They typically hear from from upper management only when things are broken or “the site is down!”

The DevOps movement has helped ease the threat of burnout by more clearly showing the link between IT functions and business value. This visibility has brought new recognition of the role’s importance, bolstered when such metrics as availability and application response time become critical performance indicators for the business.

Still, it can’t hurt for people in management or leadership positions to make time to proactively thank the people who are responsible for keeping your planes flying. Recognition and empathy can accomplish one of the report’s key suggestions to avoiding burnout: building a supportive organizational culture.

There’s lots more in the report, of course, including findings that continuous delivery can be applied to any properly architected system and the increasing importance of microservices architectures. The report also delves into the link between architecture and developer productivity, presenting evidence that diverse teams create better business outcomes.

To read the report yourself, download the 2015 State of DevOps report for free here.

This article originally appeared on the New Relic Blog, by Stevan Arychuk.