How To Predict Productivity in Agile Teams With Statistics

What do you say when your clients ask: 'So, when will the project be done?'

If your team ends up living in the office to meet the deadline you are doing it wrong. Instead of trying to determine the future, use data analysis and statistics to give a probabilistic prediction, based on historical data collected from your own process.

Statistical analysis takes the guesswork and deadlines out of estimation and replaces them with reliable, data-backed projections of any team’s productivity.

It’s time to make use of the power of statistical data analysis and here is how.

1. Collect Meaningful Data About Your Work

If you want to make data-based calculations, you need to collect that data meticulously first. Unlike paper, workflow management software can track AND store all the data about the progress of your tasks and how much time they spend at different stages. A digital platform can also allow you to compare sets of information that are months and years apart. Unfortunately, that’s not something sticky notes on a wall or any other physical medium can offer. Make the most of the available software and start tracking the key metrics of your productivity.

2. Track Cycle Time First and Foremost

How much time did a task or a feature spend in your workflow? Start measuring the time it took for your work units to go from requested to done. This seemingly simple metric is the first building block of all the other productivity metrics and can help you discover a lot about your own process.

3. Identify Queues to Calculate Efficiency

To get an idea about your team’s productivity, you need to track how much time out of the total cycle time the task has been actively worked on. To do that, monitor the time that your work units spend in queues. The difference between the two is 100% productivity, the ratio between the two is your efficiency score.

4. Measure Throughput With Efficiency in Mind

Measuring throughput is not rocket science - count how many tasks are completed that meet your quality criteria within a specific number of days and stack them up. Simple, yet very important. Looking at throughput with the efficiency scores in your hands reveals the potential for an improvement in how many days it takes for you to deliver value to your customer.

5. Eliminate Bottlenecks and Keep the Workflow Predictable

Use efficiency and throughput metrics to track down the bottlenecks in your workflow. Where do your work units spend the most time? Is it a stage in the workflow at which they are actively being worked on or a stage at which they are waiting? For a smooth flow of work, you would want to see work items balanced out among various process stages in order to avoid bottlenecks, inventory, or the build-up of a large queue.

Bonus Tip: The automation of certain parts of your process can seriously reduce the number of workflow issues by taking care that low-level tasks are moving along, work handover transfers smoothly, and you get signals about inefficiencies before they cause a halt in the flow.

6. Use Your Historical Metrics

All you need for a valid mathematical productivity estimate are historical values of cycle time, efficiency, and throughput for each period of time in your workflow. Your tools might be offering a flood of data about your productivity, but you only need these three metrics to make reasonable conclusions about your process. Use these for continuous analysis and improvement.

7. Make Probabilistic Predictions Using Statistics

To make valid statistical projections, use all this data as input for mathematically sound statistical models like the Monte Carlo simulation. Take a set of historical data about productivity, covering a stable period of 2-3 months in your workflow, and put it through the simulation to get a better idea of a probable outcome. In Monte Carlo, the data set is used as a data matrix for algorithmic analysis using tens of thousands of randomized data trials, which are then used in a Probability Distribution Function.

The outcome is a graph that gives you a statistically significant probability-based prediction. The insights in the example below conclude, with varying degrees of certainty, how many tasks your team can get finished in a month, or how quickly the team processes a specific number of tasks. Having statistical historical data means that, no matter the size of the task, you can still make a very strong probabilistic prediction and not just a subjective estimate.

So, When Will It Be Done?

Based on previous work history, the 38 remaining items will be completed in 13 days or less with 85% certainty.

It used to be that answering the ultimate client question always required one part experience and two parts guesswork. Today, we finally have the powerful business analytics tools that let us drop the guesswork and replace subjective experience with true data analysis.

It’s time statistics, and not guessing, become the core and main reference in work estimation efforts. All we need to do is to track the right things and let formulas take care of the rest.

Welcome to the Age of Data.