Test Estimation Techniques (If You Must...)

I was completely confused the first time I had to estimate how long testing work would take. It seemed so easy for most people. A big feature got a day or a day and a half, smaller features got half a day. I couldn’t even figure out what we were estimating. Was it the total time it would take to call something done including researching bugs, reporting, waiting on new builds, and retesting? Was it the amount of time something would take if everything was perfect in the first build?

I’m still confused about what what test estimation is really estimating, but now I have a few tools to help.

Let’s take a look.

Weather and Negotiation

A lot of things happen in the span of an 8 hour work day that aren’t software testing — email, coffee breaks, daily stand up, lunch, and meetings….so many meetings. One tactic called Yesterdays Weather helps sift through all the things that aren’t testing so you can move from estimating to a law of averages.  Let’s say one ‘batch’ of work is an hour long. In that hour, you are doing testing work completely uninterrupted or as close to it as possible. Some days are more productive and some days you feel like Sisyphus pushing a boulder up a steep hill. But, on an average day, you might get three of these one hour sessions done. Over the course of a normal work week, you might get 15 hours of testing work done. That number probably looks small on paper, but take a look at where your time goes and you might find it is not too far off.

The big hitch for yesterday’s weather, is that it requires change. Individual testers on the team have to start looking critically at how and how much they are working, and management has to start reporting in completely new ways. There is also the matter of collecting a few releases worth of data to see how much testing is actually getting done. I like to use a method based on informed negotiation.

Testing is like a gas, it easily expands to completely fill whatever container you try to stick it in. Let’s say I’m working on a fairly normal two week sprint cycle. Parts of features come in early, but in the last week there is an avalanche of new work to do. No matter what my original estimate was, if I even made one, that deadline is coming closer each day. Customers want their new functionality, and the development team needs to move on the the next thing.

Testing Choices

As soon as I start testing, there are some choices to make. Sometimes I get a new feature and things just go smoothly. I figure out how to use it fast with minimal documentation, it is easy to get information and figure out whether or not something is a bug, and when I do find problems the developer gets them fixed quickly. I could keep testing these till the end of the release, but usually feel like we have enough info early on.

Negotiation comes in when things go sideways. More than once I’ve had a new feature that looked easy enough on paper, but ended up taking time set up or even to figure out how to use it. The initial estimate got blown away by the work I had to do to start testing, and then again when I start digging in. As soon as things slow down I start the negotiation.  After the first day, I can start letting people know that things are moving slowly. That negotiation might mean offloading other features I was supposed to work on so I can focus efforts, asking to pair for a day or two with the developer that wrote the feature, or in the worst case start lobbying to get the feature out of the release.

Come Prepared

Negotiating worked well when I came prepared with information. That might be a growing list of important bugs, examples of what has slowed down the testing, or just the pile of work that can not possibly fit into the sprint. You can’t negotiate without leverage.

I’m sure you have noticed that estimates are off more often than they are on. I set out here to write a piece on how to make negotiation easier and ended up with something that gives alternatives. Figuring out how long it might take to test the next feature is a reasonable question. But, we might all be better off if we can make those guesses based on how much we got done in the past, or with a good feeling about how things are going now. With those tools, maybe we don’t need to worry so much about the next estimate.

What are your best tips for test estimating? Share them in the comments section below.