What does it mean for an estimate to be right?

What does it mean for an estimate to be right? Does that mean that later actuals had the same numerical value? That the project length, or cost, or end date was the same as the estimate? Is it an indication of the “correct length of the project,” whether or not the project is done in that time? Or is there some better definition of “correct estimate” that we can use?

If we choose conformance to actuals as the definition for the “rightness” or “goodness” of an estimate, we’re certainly encouraging overestimation. It’s easier to overestimate and then waste effort as needed to be “accurate” than to underestimate and try to hit a possibly impossible target. Those who ask for estimates using this definition know this, so they are likely to arbitrarily cut the estimate in order to put pressure on development and prevent padding. Of course, those creating the estimate notice this tactic right away, and have incentive to pad even more.

Once upon a time, I was working on a hardware development project. We were breadboarding a design for a custom Large Scale Integrated Circuit using off-the-shelf logic ICs. As we were preparing to prototype our second revision of the design, we calculated how many of what chips we would need. Then we added another ten percent for chip failures, and some more of certain chips we might need if we had to expand the design. Knowing that the boss had a tendency to cut parts orders in half, we then doubled the amount before taking it to him. Unbeknownst to us, he had just come from a meeting where our project was declared to be top priority. To better support us, instead of halving the order, he doubled it. That’s right, the order was placed for four times our best estimate plus padding for contingencies. We couldn’t, of course, admit that we had doubled our estimate. So, we said nothing. The crowning blow came when, a few weeks later, the project was cancelled before the parts were all received.

This story illustrates the sort of dysfunction we create when we play games with estimates rather than treating them as what they are. Estimates are not promises. Nor are they controls on what development teams can do. They are rough calculations that allow us to make decisions before we have all the facts. They may be gut-level seat-of-the-pants guesses based on past experiences, or they may be carefully calculated based on considerable knowledge. In either case, they are the best answer we can give based on the current knowledge and the amount of effort we wish to spend on them.

What, then, makes an estimate right?

1. It didn’t cost more than it’s worth. If we spend a lot of time and effort estimating something that won’t change any decisions, then the estimate is wrong because it’s all a waste. If we estimate an entire project by breaking it down into hundreds of small pieces and estimating all the pieces, it’s likely more work than the resulting estimate is worth. And if the result is close to the decision threshold, we’re probably working on something of too little value.

2. It errs on the appropriate side. The calculation of ICs needed for the breadboard was intended to err on the side of too many, so that we wouldn’t run short and risk delaying the project. If we’re estimating time so that we know when to start other activities that need to be delivered with our project, we might want the earliest time we have to reconsider when to start these activities. If we’re choosing how much work will fit in our next iteration, we might want to be wrong in both directions an equal amount over time so that our long-term projections are more accurate.

3. It has an appropriate level of precision. If we’re estimating what we can get done in the next year, we don’t want to be working at the level of minutes and hours. False precisions is just noise. When I was a poor college student doing my grocery shopping, I estimated to the nearest dime because I found it embarrassing to have to remove an item after it was rung up. That high level of precision was important to me. The shopping cart held only a dozen dollars or less, though, not a hundred.

4. It’s the right thing to estimate. If we estimate the amount of programming time for a project, but ignore significant interruptions for other work, we’re not going to have a good answer for the elapsed time of the project. My estimate of the cost of the grocery cart contents in my college days was the right thing to estimate, because when I reached my limit, I could make substitutions in my menu plans to fit.

These are some of the things that make an estimate right, as an estimate. You can probably think of more, and I’d love to hear your suggestions.