Software Rhythm Part 1: The Opening
I’ve been thinking about doing a series of posts on this for a while. For some reason I’m inspired to write it now. I’ve been involved in a number of software companies and projects of a variety of shapes and sizes but I find there is a similar rhythm across them all.
I’m going to divide the software release cycle into three phases and write one post on each. The three parts (named after the phases of a chess game) are:
Every software project I’ve seen has these aspects to some degree. The scale can vary from days to months, the formality can vary from casual to anal, and the process can be all over the map, yet I’d contend most software developers would recognize these phases.
In successful teams, this rhythm is tangible. You can ask anyone on the team, and they know where they are in the cycle, what activities are going on, and what comes next. If people have different views of where they are in the process or are completely unaware of the existence of the process, the team is probably dysfunctional and likely to fail. It’s kind of the difference between a marching band and a bunch of people wandering around holding instruments and farting.
Sometimes I struggle to describe this rhythm to people not involved in creating software. I might announce that we’ve released a new version of the product to my wife and she’ll say “Great, so you’re done?” Err…not exactly. Just moved to the next cycle.
In many ways, it reminds me of the turning of the seasons, which is something I never used to really think about. Now I relish the rhythm of the year and the feel of each passing season, even if it makes me feel like an old fart. I love watching my kids sail through it oblivious, delighting in every holiday, weather, and birthday that comes along.
But enough of the philosophizing… here begins part 1: The Opening.
What do we build?
The opening phase of a release cycle is consumed with figuring out what we are going to build. Generally the inputs to this process come from a variety of sources: product managers, sales, engineering, customers, etc.
Some group of people (usually led by product management) has the ultimate decision on what set of features will be included. It’s an inevitable fact that release dates are fixed - whether to match business cycles, financial calendars, holidays, boss’s vacation, whatever. That bothers some people but I gave up worrying about it years ago. If the end date is fixed, it’s necessary to choose a feature set for the release that can fit in the time available. 1
To figure that out, the engineering team needs to be involved to do some up-front analysis and provide rough estimates for each feature. To be able to size features, you need certain information from product management: requirements and use cases. The fidelity of these will vary greatly from organization to organization. One of the most important things you can do is to make sure that everyone involved in planning agrees on what you are building. If not, in the words of Walter Sobchak in the Big Lebowski, “you are entering a world of pain”.
Sometimes, for new or innovative features, you may just have a problem and no known solution. In that case, you may need to do some spikes to determine some options and choose one. The more innovative the feature is, the more time you will need to evaluate it. In fact, you will likely be doing this analysis throughout the release. It may even fail. That’s the risk of doing innovative work.
Once you have picked the actual release features, it is absolutely essential to assign priorities to your feature list, preferably ranking them from most to least important. This is necessary because those estimates you created suck. That’s not your fault - all estimates suck. Ranking them now makes it clear what will be dropped first when it turns out you forgot that you need to rewrite a bunch of stuff when you upgraded the version of some library to get a new feature.
How do we build it?
Once you’ve chosen a set of features and prioritized them, you need to do some design work. You may actually have done some of it already when estimating the features. I tend to favor light-weight, no cruft design docs, preferably just 2-3 pages per feature. I don’t like design docs that spell out every detail of the implementation. Do that when you write the code. The design is for flushing out the details of what you’re building and discovering what you didn’t already know.
You also need to determine a schedule for who is doing the work and how long it will take. Now again, you should remember that your estimates suck so don’t put too much stock in whatever you lay out. The key here is to build a schedule you believe is possible.
The most commonly used tool when building a schedule is the Gantt chart. I loathe Gantt charts (see Part 2 of this series for why and some better tracking options) but for an initial cut at whether you have a believable schedule, they’re not bad. I’m especially in favor of them for this if I can sucker a project manager into creating them for me.
Personally I find I can get the same amount of information by creating what I call a “blocking chart”. It’s just a table with weeks on one axis and the people in your team on the other. 2 Indicate in each block what feature the person is working on. Based on your estimates, you should know roughly the time available and you probably have a good idea who is actually working on it too. The who is very important of course, because people’s knowledge, skills, and abilities vary by 10x. Fortunately, as a person that knows the people in your team, you can make these calculations by just playing the idea in your head. This technique is probably worth a post in itself…
The most common thing to screw up in the Opening is to under-estimate the amount of time it will take. A lot of people schedule one or two weeks to do analysis and design. That’s not enough, unless you have timeboxed only 4 weeks for your release. 3 It’s a good rule of thumb that you should spend a third of your release cycle figuring out what you’re building, a third building it, and a third releasing it. Those proportions will vary quite a bit based on your processes and environment, but I find the tendency is to make the Opening activities too short and the End Game activities too long.
If you short-change figuring out what you are building, you’re going to fail. You will discover at some point that you’ve built the wrong thing. If you’re lucky, you’ll discover it before you release and you can correct it and you will fail merely by being late. If you discover it after you release, you will fail by delivering something that isn’t what people want. The trick of course is to figure out what to build AND build it before what you need to build changes. :)
That trick of course, is what agile methods are all about. An agile process gets out of the way so you can translate a software need to actual software rapidly. This gives you short periods of clarity that can be repeated to spiral in on what you really need.
If you are using short iterations in an agile methodology, I think that everything here still applies, but at two separate levels. First, my opinion is that you still need an opening phase at the beginning of a release cycle to set the overall direction (others may not agree). You may choose to just map out a series of features (*cough* stories) and their initial ordering. Second, you still need to do some form of planning and design in each iteration that will look very similar to the opening phase, just in compressed form.
1 If the engineering team is being handed both dates and a feature list then the only variable they can affect is quality, and it’s gonna be bad. It’s your responsibility as an engineering team to push back if this happens to you. My experience has usually been that there is some way to compromise if everyone goes into the discussion with the idea that there is actually a compromise. Sometimes that means looking at whether the parts of a feature that are most costly to build are the most important to the business. If not, you might be able to pare down a feature or come up with some way to stage the feature.
3 By the way, I find a release with 4-6 weeks of dev is what I like the best. I can hold the schedule outline in my head for something scoped to that size. I expect to spend at least 2 weeks of that period doing planning and design. Realistically the planning lasts longer as a few people may start the process before the prior release is out and/or may bleed late for a subset of the features.