Thoughts On the 10th State of Agile Survey
The new State of Agile survey has seen pretty consistent results compared to last year, but some of the data and the samples used seem to be less than perfect...
Join the DZone community and get the full member experience.Join For Free
Just recently I reviewed the 9th Agile survey (read about it here) and now the 10th version is out (you can access the survey here), so I thought I'd provide an update. Given the results are not that different from the previous year, I will write this post a bit differently. I will review some of the key results and then provide some critical thoughts on the survey.
Part 1 – Review of the Results
Our Delivery World is Complex (Multi-modal, Distributed, and Outsourced)
Looking at the results about company experience, 53% say that less than half of the teams in their organizations use Agile, and only 9% say that everyone is doing Agile. So clearly we live in a multi-modal world. 82% are working with distributed teams, which is larger than I expected but reflects the largely distributed nature of IT projects these days, and nearly 70% use outsourcing for their development projects. This complex environment is normal. I find it interesting how many clients tell me they are “special” because of these characteristics: they are clearly not.
What Are the Typical Benefits of Agile?
The report highlights the following three to be the top answers (which have been the same last year):
Ability to manage changing priorities
This sits well with me as the usual suspects that we have to debunk are not in this list, e.g. faster time to market (only on rank 6) and cheaper delivery (not in the list at all).
How Do You Measure the Success of Agile?
The answers to this are also stable:
Good answers, I think this reflects what success looks like well. I guess these are somehow measurable and provide some success measures for Agile.
How Do You Scale Agile?
We see a significant jump for SAFe here from 19% to 27% and I am not surprised. It certainly has a lot of momentum in the industry and provides practical guidance that I leverage in my daily work. Other frameworks are really not present (DAD, LESS,…) and have less than 10%.
Of the top 5 tips for Scaling Agile, at least two are in my top lessons learned: Consistent process and practices, and Implementation of a common tool across teams. I agree with the other 3 tips as well: Executive Sponsorship, getting help from someone with experience like an Agile consultant and creating an internal support team.
What Tools Do You Use?
Really not much movement here. The previous favorites remain on top: Jira, VersionOne, and TFS.
What Makes Agile Fail?
Culture has been called out as the main reason this year, and it is hard to argue with this. However, it's difficult to address directly. The other reason is easier to address: lack of experience. This means that we should make sure experienced Scrum Masters and coaches support new projects. Lack of management support and lack of support for the cultural shift are next, something we need to work on as an industry. Then a very addressable one still gets 38% of the votes: inconsistent agile practices and processes. This we need to fix. I don’t understand how so many people still allow Agile projects to fail because they think Agile is a freeform exercise. At the end of the day, there is a project/outcome to be delivered and too many coaches still shy away from that accountability. If you work with Agile coaches, make sure they have skin in the game so that they balance learning with delivery responsibility.
Overall I like the results in the report, and they're consistent with the previous year which is good to see.
Part 2 – A More Critical Look at the Survey
I like the results of the survey and got some interesting insights, but how valid is the survey? Do I understand the mechanics behind it well enough to use the data?
Let’s look at a few pieces of information that should make us think about the data before blindly quoting it:
- First of all – who is the sample audience? Clearly the people responding to this have somehow heard about Agile as it is unlikely that others respond to an Agile survey. The people who respond are self-nominating, so they might be more on the extreme ends and feel they have something to contribute. It is not a random sample of people working in IT. This influences the data to some degree I would think.
- 1% said that Agile adoption has failed in their organization. That number seems very low and might be because the people who've seen it fail are not responding, or it could be because the definition of failing is actually quite difficult to understand. Does failing mean you are completely going back to Waterfall, or does it mean you had mostly bad results from Agile? Not sure what my definition would be, so perhaps the lack of definition causes people to not use this value in the survey.
- Agile techniques used – Now here I really start to struggle with the values and figuring out how this information should help me. Do only people who use Agile respond to this or does everyone (even those using Waterfall)? Daily Stand-ups and prioritized backlog seem pretty fundamental to me and except in rare cases I would expect Agile projects to use these. The numbers seem low on that basis. But wait, it gets much worse, only 54% do iteration reviews/showcases, and only 45% have devs and testers in the same team. I would argue that if you don’t have those two you are not really Agile. I defined Agile in another blog post as “ONE team delivering an INCREMENT of a product (at least product tested) within ONE iteration”. The problem here is that one cannot relate the info back to the methodology used and hence it becomes difficult to derive insight from this data.
- Success measures – Velocity is the largest one. I was hoping we got over the velocity race, but clearly it’s still on. The success measures are very contextual and I would caution anyone to use this without thinking. Take for example “Planned vs. Actual stories” – it could be very positive to deliver more stories than planned or even less stories (as long as the most value is delivered). Context is unfortunately king for metrics, so don’t use the information from this survey blindly.
Published at DZone with permission of Mirco Hering, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.