It is interesting to see that metrics has become such a controversial topic in the agile community. Metrics should, in the best of worlds, be used as a tool to support informed decision making. Using this definition, it's hard to understand how it can cause such strong negative reactions. To explain it, we have to listen to the disapproving voices and digest what is being said.
WHY ARE METRICS CONTROVERSIAL IN THE AGILE COMMUNITY?
As with so many other things, it seems like the metrics phenomenon is blamed for all the pain and inefficiency it causes when wrongly applied; And this happens ALL the time. Especially in organizations that recently have transitioned from traditional ways of working to becoming more agile.
Many organizations that have gone through an agile transition still carry a heavy legacy in the form of metrics that, at best, were relevant for the old ways of working. (“Time spent” is an example many can refer to…) This legacy of metrics were used to support decision making in a way that was needed earlier, but has little to offer the agile organization’s new instances for important decision making.
DECISION-MAKING IN THE AGILE VS. TRADITIONAL ORGANIZATION
What do we mean by “new instances for decision making”? Unlike the earlier non-agile era, a heavy part of decision making finally resides where it should; with the individuals that are closest to the development work. So, “new instances for decision making” refers to the team/squad/scrum-team of developers. In earlier days the decisions happened higher up in the organization and metrics were to a larger extent defined by managers and pushed for top-down. At the top, people needed data from the bottom to be able to carry out informed decision-making. Later, when the organization underwent modernization to become more agile, all the metrics that should have been killed were not and many developers are still forced to maintain these metric artifacts for reasons that are insufficient.
In agile organizations the team is empowered. The team makes decisions about many things that were previously steered from the top. Questions like “how should we execute this work?”, “who should do what part?”, “in what order should we undertake this?” and “can we go home an hour earlier today?” resides with the team and they themselves are responsible for carrying out the informed decision-making needed to answer these questions. If they feel that their decision making becomes better by being supported by metrics they should of course use metrics. They should be empowered to track whatever metric they want to make better decisions about the things that resides within their area of responsibility.
METRICS IN LARGE-SCALE AGILE ORGANIZATIONS
When looking at large agile organizations that consist of “teams of teams”, the metric question becomes very interesting. How these organizations are structured depends on what framework they have chosen for carrying out agile at scale, but what they often have in common is that they have distinguishable “levels” and that certain responsibilities, certain decisions, reside on each level. One clear example of this is SAFe, which talks about the portfolio, the program and the team level;www.scaledagileframework.com. This framework describes what decisions are being made at each level and also suggest a set of metrics that could be used to support each level’s decision making.
Figure 1. Decision-making and the need for metrics occur at all levels in an agile organization. The trend, compared to the traditional organization, is that the teams carrying out the development are more empowered in agile organizations. Hence, the need for metrics is spread across the whole organization rather than residing at the top only.
The million dollar question becomes what decision making, and hence what metric, belongs at different levels at large-scale-agile organizations. Answering this is not trivial as large-scale-agile is carried out in a great variety of ways and the responsibility of different levels or roles in for example Spotify’s model is nowhere near the once described by eg. SAFe. That said, there are some metrics or measuring practices that could qualify for being relevant across different types of large scale agile set-ups and I will later on return to what these metrics can be.
But before we get to concrete metric recommendations, it makes sense to state some general guidelines for large-scale-agile metrics;
- An agile organization should continuously improve its development processes. Metrics should be used to allow for process improvements in an agile fashion.
- Important decision making happens everywhere, across several different levels in large agile organizations. That’s why, each level should use the metrics they need to support the decisions they are responsible for.
- The metrics you are tracking should be owned by those who need them to make informed decisions.
- The metrics need to be actionable, meaning the information gathered should be able to lead to a concrete action towards reaching a prioritized goal.
- All metrics should be iterative and evolve over time to always align with the priorities of the metric owner/s.
- Metrics should be open and transparent to everyone.
- The current set of metrics should be kept very limited and surveyed closely. Too many metrics at the same time will dilute your most prioritized goals and decisions.
- Metrics of secondary importance should halt or be put aside.
- Sometimes, individuals other than those who need the metrics have to be engaged in data logging, gathering or some type of administration. In these cases, information on exactly what decision making the metric supports, who needs it and why it is prioritized needs to be shared and communicated.