Release Planning and Predictable Delivery

many organisations wrestle with the seaming incompatibility between agile and release management and they struggle with release planning and predictable delivery.

the incompatibility between predictable delivery and agility is fictitious and while usually created by an organisation and structure that is unwilling to let go of the old ways and embrace the tenants of agile it can also be the result of a teams fervor to divest themselves of all things that smack of prior planning.

there is a lack of understanding that agile and the path to agility is far more than just a change in the way that you build software, it is a fundamental shift in the way that you run your business. much like the lean movement in manufacturing, companies that embraced it wholeheartedly were the ones that ultimately see the competitive edge that it provides. if one is unwilling to let go of the old ways then one can’t attain the value of the new.

this change will take hard work and courage as the fundamental transparency required to inspect and adapt effectively is at odds with the measure of the past. the lack of predictability of software development is the key to understanding the new model.

why is software so unpredictable

all software development is product development.

in lean manufacturing we can optimise the production of pre developed products through the nature of its predictable production. each widget takes the same amount of materials and time to produce so any changes that we make to that time or materials can easily be qualified and the benefit demonstrated. manufacturing is predictive.

with software everything that we create takes its own amount of time: you can really only know how long something took after it has been completed. even in manufacturing if you asked an engineer how long it would take to develop a new widget they would not be able to tell you with any certainty. once they have developed it however they can tell you exactly how long it will take to make each one. in software development we are always doing new product design therefore we have no certainty…and this often results in chaos.

all is not lost however as we can, by looking at our history of delivery for similar things, make a pretty good forecast …

the best thing we can then do is to expend effort to make that forecast as accurate as possible while accepting that more time spent planning does not necessarily affect the accuracy of that forecast.

figure: diminishing returns from agile estimating – estimation approaches

ultimately software development is a creative endeavour and has the same lack of predictability that painting a picture, writing a book or making a movie has. yet movies get made all of the time. how can that possibly be! well they have a director (product owner) that has a bunch of money and a plan for delivery, a producer (project manager \ scrum master) to make sure that everyone has the skills, knowledge and assets available at the right time and place and one or more units (development teams) that have all of the skills necessary to turn the directors ideas into a working movie. they create storyboards of what they expect to create so that they can run it past the stakeholders and get feedback. they take those storyboards to the units who collaboratively work together with the stunt, prop, lighting, camera, sound and wardrobe crews to get estimates and costs and ultimately coordinating to create the movie.

making a movie is just like building software, you need a budget, you need a plan, and you are trying to reach a ship date. and just like building software you have to make money at the end of the day so that you can do it all over again.

accept the lack of predictability

while i hope by now you understand that the lack of predictability is part of the nature of building software there are many things that we can do to lessen the impact of that chaos. indeed if you were to estimate all of the discreet things that you need to do to achieve a goal (lets call them backlog items) in small, medium and large what would your standard deviation of actual hours be? i would wager that it is fairly large. so large in fact that at least half of all mediums would be more accurately classified as large. but that reclassification can only be done with hindsight.

this is normal for organisations that move towards agility as the transparency that it brings uncovers these sorts of problems. in order to increase the accuracies of our forecasts there are a number of simple activities that we can perform. these activities, while easy to understand, are very hard to do as they require a culture shift within your organisation as well as the courage of the participants to make them work.

focus on continuous quality

most software lacks quality for one simple reason. you can’t really see the quality like you could with a table or a painting. i am not talking about the quality of the user interface, but the quality under the covers; the quality of the code.

if you put developers under pressure to deliver they will continuously and increasingly cut quality to meet deadlines.
-unknown

a lack of quality of the code results in a increase in technical debt (or more accurately an unhedged fund) which in turn results is two nasties. the first is the teams  increasingly have to spend more time struggling with the complexity of your software rather than on new features. if you are still pushing your teams to deliver the same feature level every year you are only encouraging them to cut more quality and thus incurring technical debt becomes a self-fulfilling-prophecy. the second is an increasing number of bugs found in production. bugs found in production also directly impact on the amount of features that the team can deliver and any bug, no matter how small, costs ten times ad much to fix in production than it does in development.

there is only one way to deal with technical debt:

1. stop creating it
2. pay a little back each iteration
3. goto #1

there are a number of strategies that can help you both stop creating and start paying back technical debt:

• sufficient requirements – if your backlog has things in it that are too big or to vague then your team will not really be able to understand them and this in turn creates a multiplier for uncertainty. follow the invest (independent, negotiable, valued, estimable, small, testable ) model for every single thing that you ask the team to deliver. if you invest in you backlog in this way you will find it much easier to deliver the contents and thus predict that delivery.
• team chooses what they can deliver – this implies that the development team can reject any item on the backlog that they do not understand. if we accept that every development team is trying to do their best to deliver for their product owner then the only reason to reject anything would be if an item is too big or does not have enough detail to understand.
• definition of done (dod) – along with having sufficient requirements the single biggest blocker to predictability is a lack of common understanding of done. done for a development team should equal what it means to complete an item with no further work required to ship it. as a reality check you may need to start with less than this and then build it up as you pay back your technical debt. technical debt is the reason that your dod is diluted and once you have a representative dod your development teams predictions will get way more accurate.
• fixed length iterations – if you have variable length iterations you can’t be sure what you can do in a particular timeframe. how much decomposition do you need to do to the backlog? how much can the team deliver in a single iteration? you can’t be sure unless you have fixed length iteration.
• no separate teams – this means no separate test teams, configuration management teams and definatly no separate maintenance teams. its hard for folks to grasp, especially with the recent focus on devops but if you have separate teams then why would your development team, those best placed to fix any problems, care about the problems of other teams. the most successfully organisations at creating software have development teams that own the entire application lifecycle (amazon aws | visual studio.)
• manage dependencies – managing dependencies is hard and my advice is don’t. a development team should have all of the skills required to deliver what you want at the quality level that you want. so if you need to have productionised databases, or scripting for production deliver then you might need a dba or an operations administrator or two. this can be hard for many teams or organisations but you will have far less success creating silos like configuration management or devops. rather add those individuals that you need to the team. however if you have a dependency on a separate team, maybe you have an application upon which all of your other applications depend, then you may need another way. this is not a silo of types of individual skills, but of domain and that team just has something in their backlog upon which you are dependant. it is up to the teams respective product owners to fight negotiate over when these things get done.

if you can, do them all…

conclusion

remember that any software that you create is an organisational asset and decisions to cut quality need to be reflected in your company accounts and as such those decisions need to made by your executive leadership and should not be made by the development team.