Agile Estimating: Story Points and Decay

I’m re-reading Mike Cohn’s Agile Estimating and Planning. It's the best book I've found on this and worth reading, even if he gets too Scrummy at times, and even if you don’t agree with everything he says. Which I don’t.

For example, I don’t agree with him that Story Points are better for estimation than Ideal Days. When we do estimates, we use Ideal Days, because thinking this way is more natural and more useful, and because these estimates are more meaningful to people inside and outside of the team.

Estimates in Ideal Days Decay

One of the reasons that Cohn recommends Story Points is that, unlike estimates in Ideal Days, Story Point estimates “have a longer shelf life” because they don’t decay – the meanings of the estimates don’t change over time as the team gets more information.

If you’re estimating using Ideal Days, as you learn more about the tools and language and platform, as you have more code to build on and work with, your understanding of how much work you can do in an Ideal Day will change. You may be able to get more work done – or less, because it turns out that you need to spend more time testing or whatever. As you learn and get better at how you work and understand it better, the definition of an “Ideal Day” will change. You’re learning and getting more information, so why not use this information and make better-quality decisions as you go forward?

But this means the estimates that you did earlier aren’t as useful any more – they’ve decayed. You’ll have to re-estimate this work at some point, because you’re mixing before-the-fact and after-the-fact information.

At some point you may have to do this anyways, whether you use Ideal Days or Story Points. Because relative-size estimates in Story Points also decay over time.

Story Points Decay too

Estimates in Ideal Days change as you learn more about the solution space: about how much work it actually takes to build some thing. Story Point estimates change as you learn more about the problem space: as you learn more about the domain and the problems that you need to solve, as you learn more about how big something really is in comparison to other things.

Take Cohn’s “Zoo Points” exercise, where people attempt to estimate the relative size of different wild animals. Everyone knows that a blue whale is much bigger than a rabbit or a fox. But if you’ve never seen a blue whale in real life, you have no real idea just how much bigger it is. 10x? 100x? 500x? 1000x? What about a tarsier or an okapi – some of the rarest animals in the world. You’re unlikely to ever see one. How the heck could you know how big they are relative to a rabbit, or to each other? You don’t know, so you’ll need to make a reasonable guess and use that for the time being.

Later, as you learn more about the problem space, especially if it’s a deeply technical domain, your understanding of size and complexity and risk will improve, and could change a lot.

So your idea of whether a thing is twice as big as something else, or about 1/5 as big as another thing, whether it's worth 5 points or 15 points, will change as you understand more about what these things actually are. As you continue to estimate the changes and new requirements that come in, your new estimates won’t align with earlier ones. The kind of work that you thought to be 5 points earlier you now know is 15 points. Unlike an Ideal Day, you haven’t changed what “5 points” means, but you have changed your understanding of what work is actually “5 points” worth.

On a project that’s run for a long time, people will join the team bringing new knowledge and experience, and people will leave taking knowledge and experience with them. Which means that estimates that you made earlier, before you learned something, or when you knew something that you’ve now lost, can't be the same as the estimates that you're making now, regardless of whether you’re sizing work in days or points. It’s unavoidable – you can’t not use what you have learned, and you can’t keep using what you’ve lost.

There may be reasons that Story Points are a better choice for estimating, for some teams at least, but “decay” isn’t one of them.