And there it is — the most recent State of DevOps report, the 2016 version. If you read my previous blog posts for these kinds of reports, you will expect a full summary. Sorry to disappoint. This time, I focus on the things that stood out to me — we all know DevOps is a good thing, and this report can give you ammunition if you need to convince someone else. But I don't see the point of reiterating those. Let's focus on the things that we didn't know or that surprise us.
It is great to see that the report continues to highlight the importance of people and processes in addition to automation. That is very much in line with my practical experience with clients. High-performance organizations have a higher Employee Net Promoter Score (ENPS), which makes sense to me. I think there is an argument to be made that you can use ENPS to identify teams or projects with problems. I would love to test this hypothesis as an alternative to self-assessments or other more involved tools that might cost more but might not be more accurate and are harder to deploy.
Another key finding is the impact it has when you build quality into your pipeline and don't have it as a separate activity (e.g. no testing as a dedicated phase — I wrote about modern testing here), but then the numbers didn't really convince me on the second look. It's difficult to get this right, especially as the report has to work with guestimates from people at all levels of the organization. But I agree with the sentiment behind it, and there is anecdotal evidence that it holds true. I would love to have some more reliable data on this from real studies of work in organizations — it could be very powerful.
This year is the first time DevOpsSec is reflected in the report, and the results are positive, which is great. I have always argued that with the level of automation and logging in DevOps, security should be a lot easier. The report has some very useful advice on how to integrate security all through the lifecycle on page 28.
We continue to see a good proportion of people coming from DevOps teams, which is not surprising, as that is the organizational form that most larger organizations choose for practical reasons (at least as a transition state) and flies in the face of a "DevOps team" as an anti-pattern. Glad to see the reality reflected.
On the results side, the report uses some pretty impressive numbers on what high performers can do compared to low performers. That's great info, but I would like to see this compared between companies of a similar size and complexity — otherwise, we compare the proverbial DevOps unicorns with large enterprises, and that is not really a fair comparison. The difference is not just in DevOps then. The more detailed data shows, in my view, the limitations of the comparison and some "kinks" in the data that are not easy to explain. I am glad they printed this data, as it shows that the researchers don't massage data to fit their purpose, which is good.
I really like how the researchers tried to find evidence for the positive benefits of trunk-based development, but I am not convinced this has been fully achieved yet. The same counts for visualising work — I see the point, but the report does not give me more reason and ammunition than I had before, in my view.
Similarly, the ROI calculation is a good start, but nothing revolutionary. It's worth reading, but you won't likely find much new here — reduction in downtime, reduction in outages, increase in value through faster delivery.
Overall, it's a good report, but it's not revolutionary. It's great to see the trends over the years and that the data remains consistent. I'm looking forward to next year's version. And yes, I am writing this against the high expectations from last year, but it's difficult to have revolutionary news each year…