As we conduct more and more transactions online, the performance demands we place on IT operations departments have scaled exponentially. So have the costs of an outage: Earlier this year, Apple experienced an outage that cost it an estimated $25 million and contributed to a 1.8 percent drop in its stock price. Dimensional Research conducted a survey — commissioned by PagerDuty — of 674 companies to assess their attitudes, experiences and approaches to incident management in this high-stakes environment. The results are startling.
Outages, Noise and Missed Alerts Cause Biggest Problems
According to the survey, 52 percent of companies polled have a severe incident at least once a month. The industry has seen a proliferation of monitoring systems in response to these rates, but investment in new technology alone isn’t solving the problem. Of respondents, 45 percent said their teams work with six or more monitoring systems, yet only 27 percent consolidate alerts in one place. This suggests that IT teams are dealing with too much noise, which leads to missed alerts, outages and engineer burnout. Of those surveyed, 85 percent said they miss alerts, which could have a disastrous effect on their business. After all, 99 percent of companies said downtime affects their bottom line.
The Solution Is DevOps
Though the big picture is somewhat dismaying, there’s one bright spot: teams that have adopted the developer/operations (DevOps) ethos have a better handle on their IT incidents and suffer less downtime. By focusing on collaboration and open communication, DevOps teams bring developers and operations engineers together to deliver new functionality faster, while also minimizing the production issues introduced by those frequent changes and correcting them faster when they occur. Teams adopting the DevOps approach quickly gain confidence in how their software will behave in production, and in their own ability to quickly make changes to both the software and environment to respond to operational incidents right away.
DevOps Improves Uptime
Adopting a DevOps strategy helps companies improve uptime. The Dimensional Research survey found that 31 percent of DevOps teams never miss critical alerts. Not one. Ever. None of the other companies surveyed could claim that, even those that have partially implemented a DevOps model. Surprisingly, DevOps teams receive more alerts than non-DevOps teams, yet they are still able to manage these alerts more efficiently. Not one company with a DevOps model reported taking more than an hour to respond to an incident, and 77 percent of DevOps companies said they respond within a half-hour. In addition, DevOps teams garner a higher level of incident response satisfaction from stakeholders. Only 6 percent of respondents reported dissatisfaction with incident response, in contrast to 30 percent among companies without DevOps teams.
DevOps teams know how to translate these incident response successes into improved customer relationships. They are 30 percent more likely to be transparent with customers about critical incidents because transparency is a key principle of every stage of DevOps production. If your production ethos is all about keeping your co-workers in the loop, it’s only logical to extend this attitude to customers. And it’s becoming clear that customers are ready to reward companies that hear their voices and respond in real time.
DevOps Teams Succeed Through Collaboration
DevOps is not just a set of tools companies purchase; it represents a fundamental shift in a company’s approach to work and culture. Building a culture of transparency, trust and collaboration is difficult, and the Dimensional Research survey found that DevOps teams take three important steps to achieve their remarkable results:
Alert Aggregation: Approximately 56 percent of DevOps teams surveyed use a system to consolidate alerts, as opposed to 30 percent of companies without DevOps teams. By cutting out alert noise and bringing everyone together around a single alert dashboard, DevOps teams can more clearly see what issues are urgent and who is taking steps to resolve them. Transparency is crucial to DevOps success.
Agile Development Practices: DevOps teams are 30 percent more likely to use agile development practices. Because agile development requires incremental feature delivery and regular testing throughout the lifecycle of a product, agile DevOps teams can quickly respond to issues, roll back any errors and ensure product quality at all times. Being agile means being exceptionally responsive to customer and stakeholder feedback, which drives customer satisfaction and stakeholder buy-in.
Real-Time Communication Tools: Twice as many companies with DevOps teams use a company-wide chat system as companies without DevOps teams. Email, phone calls, and meetings will never fully be replaced, but DevOps teams know that they need to collaborate in real time and in a forum where everyone can see a record of the actions being taken.
The DevOps Era Has Arrived
Customers are online right now. They’re talking to companies and to each other on social media. They want to place orders online, and they’re not going to wait around if the website is down. The costs of downtime are rising, and maintaining uptime takes more than a handful of monitoring tools. It takes alert aggregation, agility and collaboration to keep pace with increasing customer demands. The results of the Dimensional Research survey suggest DevOps is the best way to stay on top of incidents, cut through alert noise and resolve issues quickly. Improved uptime due to DevOps practices translates directly into improved customer relationships and a competitive edge in today’s marketplace.