Product Backlogs in Practice

A few days back I came across Yuriy Zubarev's Cynical Agile and Scrum Dictionary. There are some nice, cutting observations in there, but the one that has really made me smile is:

Backlog: a land of forgotten dreams

Years ago I went to college on the Isle of Skye, off the west coast of Scotland. I learned about Tìr na dìochuimhne, and other such mystical places of Gaelic culture. These days my perception of the Lands of Forgetfulness is driven not by any loch-mist that streams through the hills to the isles, but by the forsaken backlogs I find on Scrum boards. All too often the user stories that are put on a product backlog never make it to reality. All through an entire project they can sit there, these creatures of whimsy, half-reminding a team of something which should have been done, or which might have been done, if only they had time to get round to it, or which could have been looked into, but never was. Each was put there for a reason that the team may not even remember any more. Perhaps they never really knew why. "Anyway", they tell me, "there have always been other more important things to be getting on with". The Product Backlog is the land of forgotten dreams.

What a Product Backlog should be

Seen at its simplest, a Product Backlog is an ordered list of requirements, the content and ordering of which is maintained by a Product Owner representing one or more customers. Each requirement is sized by the development team, such as by estimating the effort that is likely to be needed to complete it. In an agile method like Scrum, a portion of those requirements are taken at regular intervals and worked on by the developers who will then produce something of value. Each interval is known as a Sprint, and the portion of requirements to be actioned in a particular Sprint is called the Sprint Backlog. The Product Backlog is the set of requirements for the entire project. The responsibility for managing it lies wholly with the Product Owner, although he or she must negotiate each Sprint Backlog with the development team. Since the Product Backlog contains requirements of a given size, and each Sprint Backlog contains some of those requirements which are of a given size, the rate of a project's progress can be measured.

However, a Product Backlog is never complete because scope and priorities are always in some degree of flux. That's just fine though. We know that requirements change all the time, but by keeping a Product Backlog updated we can embrace that change instead of fearing it. With a known rate of progress the impact of such scope changes on the estimated date of project completion can be foreseen, and stakeholders can modify their plans and techniques accordingly. These three legs of transparency, inspection, and adaptation underpin an agile way of working.

When Product Backlogs go wrong

Ironically enough, the seeds of backlog decay are often sown in this very flexibility, and in the willingness of a team to be adaptable. When a new requirement is introduced there is a very high likelihood that it will be an urgent one. For example, Scrum teams aren't always dedicated to project work but very often have to provide Business As Usual (BAU) support as well. This support work will invariably trump development priorities and there is an excellent chance that BAU requirements won't be put into the Product Backlog at all. Instead they'll be fast-tracked straight through to "Work in Progress". In effect the quality of service provided by the Scrum team is permitted to vary, somewhat in the manner of Kanban. This hybrid way of working is known as Scrumban and by my reckoning it must be the most widespread Scrum variation to be found in industry today. It's easy to see that if fast-tracking is left unmanaged, the team will operate in a reactive mode and will never become a truly agile one. The backlog will fall into a state of neglect.

So whose responsibility is it to manage this unforeseen work? The answer depends upon whether it relates to the product being developed or to something else entirely. For example, if the project involves the enhancement of an existing system, it's quite plausible that any BAU fixes will be related to that specific product. The call on prioritization - to fast-track or to place in the Product Backlog - can and should be made by the Product Owner. Only that person is authorized to decide on matters of product value, and if a Scrumban approach has been agreed with the developers, they'll be obliged to provide the quality of service that fast-tracking represents. Clear, unambiguous product ownership is very important in this situation.

On the other hand, if the team are also supporting a completely different product - or set of products - as part of their BAU responsibilities, the Product Owner won't be in a position to make the call. Sometimes it's only the team themselves who can triage BAU requests coming in, although if they are driven through a customer support help-desk it may be done for them. In either case the team will have to appraise the Product Owner of this fast-track work. At the Product Owner's discretion, the Product Backlog can then be revised, customers notified, and new priorities set which reflect any projected impact on schedule. Yet again clear product ownership is important if backlog management is to happen. Without it a Product Backlog will decay and fall into lost remembrance.

Three tips for avoiding decrepitude

Here are three key measures that will have to be taken to stop a Product Backlog from corroding:

1. Clear Product Ownership. A development team is responsible for what it accepts into each Sprint Backlog, but ownership of the Product Backlog always lies with the Product Owner. That's the person who must understand the business value of each item, and who can order or prioritize them accordingly. A Product Backlog should have exactly one Product Owner. If there is no Product Owner at all, then the Product Backlog cannot be maintained in an effective and businesslike manner. If there are multiple Product Owners arguments over priority can arise or ownership can be weakened in order to avoid such conflict. As the Scrum Guide points out, "Product Ownership is not a committee". A limited degree of compromise can be allowed, such by allowing a Product Owner to be represented by proxy, but this must always be treated with caution. Whoever represents the Product Owner must be fully able and empowered to make decisions regarding product value on their behalf. This is the person who must keep the Product Backlog up-to-date, and who can account for each item on it so it does not become a forgotten dream.
2. Frequent grooming. The Product Owner and the development team should review the entire Product Backlog at least once per Sprint. This can be done as part of the Sprint Review although it is also acceptable (and increasingly common) to arrange dedicated sessions for this purpose. The idea is to give the team the opportunity to provide or revise estimates for each item on the backlog, and to identify any issues or ambiguities that might forestall Sprint Planning. The process of grooming a Product Backlog can be instrumental in keeping it up to date and relevant, and fresh in the minds of all stakeholders. Team grooming becomes especially important when product ownership is shaky.
3. Collaboration with other teams. Projects rarely exist in isolation, and are more typically components within a larger portfolio of changes that reflect an organization's strategic goals. Product Backlogs can therefore be highly interdependent at a portfolio level. This means that teams will need to collaborate with each other, both directly and through their respective Product Owners, to make sure that release trains are synchronized and in accordance with compatible Definitions of Done. Failure to collaborate in this way can lead to the disintegration of Product Backlogs that no longer reflect the intended strategic direction.

Conclusion

A Product Backlog is far more than just an ordered list. There are a range of behaviors that must come together for one to be effective. These include clear product ownership, a team commitment to review and estimate, and ongoing collaboration with the wider enterprise. A Product Backlog will not manage itself. It must be subjected to diligent curation if it is to avoid the journey into the land of forgotten dreams.