Story Branching and Continuous Integration: a swords-to-plowshares tale

I'll confess to being a little intimidated by the prospect of writing this blog.  See, I haven't been active as a coder for a couple years and haven't yet worked on a collaborative Git or Hg project.  (It's on my to-do list... just haven't swizzled it in yet.)  But lack of hands-on experience notwithstanding, I'm really excited about a growing trend amongst Git and Hg teams.  No, not Pokemon Exception Handling.  I'm talking about story branching. It's all the rage at Atlassian (my employer - insert disclaimer here, yada-yada-yada), and I want to share our approach with you.

The basic idea of story branching, sometimes referred to as "issue-driven development", is that you create a development branch for each and every JIRA issue you implement.  Bug fixes, user stories, spikes... they all get their own branch.  Madness, you say!  And I would agree with you if we were still using SVN, in which branching and merging is not exactly lightweight.  But branching in Git is very lightweight and merges don't lock up the entire repository, making this crazy idea quite practical.

In the not-so-recent past, continuous integration in the context of story branching was considered so impractical as to be outright incompatible.  Who has time to manually configure a CI scheme for dozens of active branches that will only live a couple of days?  Nobody - that's who.  And story branch-loving teams would frequently encounter *ahem* "surprises" when merging work onto the main code line --master', in Git-speak.  But recent versions of Bamboo have essentially removed that overhead and allowed these two 'frenenmies' to become genuinely harmonious partners in our development lifecycle.

Story Branching 101

Let's break it down.

To get a feel for the story branching workflow, we'll follow the life cycle of a typical JIRA user story, starting from the time when the story is picked up by a developer.  At this point, the developer chooses a recent stable point on master and creates a branch from there.  Naturally, all Atlassian teams build using Bamboo which listens for new branches created in our repos.  When the story branch is detected, Bamboo finds all build plans associated with that repo and creates what we call a 'plan branch' - essentially, a clone of the original plan that is pointed at the new code line.  The branch detection and plan branch creation happens automatically for Git, Hg and SVN repos - so whatever CI scheme the master branch is under has been applied to the story branch without the humans having to even think about it.

Next, we go into development and testing cycles on the story branch.  The developer is making local commits on her workstation.  In Git, this feels the same as committing a change in SVN: you add a commit message, etc.  But since everything is happening locally, the transaction is as fast as a simple save.  All the benefits like revision history and comments, without the downside of network latency.  When she reaches a point where she wants to run tests against her work, the developer pushes those local commits up to the repository - still on the story branch, of course.  Bamboo sees that there's been a change, and starts building the corresponding plan branch/es.

Even this simple thing is a big leap forward compared to where CI was 8 months ago because what used to require manual administrative overhead (setting up a plan to build against the branch), we now get for free.  And it's huge for a team's velocity because it allows us to catch most bugs while the work is still isolated on a branch.  Nobody else's workspace is polluted, so nobody else is blocked.  Two developers working in the same area of the code don't have to un-tangle each other changes in order to fix bugs they introduce.  And master remains in a releasable state all the while.

To help find conflict before merging to master, the developer can choose to have Bamboo merge master into her story branch each time she pushes changes to the repo, then run through the CI plan against the merged code. If the plan passes, Bamboo will commit the merged code to her branch. This prevents "drift", and eases the merging situation in general by performing small, frequent merges.  This feature is most effective when developers have sufficient privileges in Bamboo to toggle auto-merging on and off as they see fit.   As Bamboo's tech lead says "If I'm half way through a feature, and everything broken, there's no point pulling in other people's changes that will make it more broken."  Testify, my brother.

When the story is complete (read: implemented, code-reviewed, and passing tests at all levels of the application stack), it's time to merge the story branch onto master. The story branch is closed out, the developer picks up her next issue to work on, and nants ingonyama, the circle of life begins again.

Story Branching 102

Some teams are taking this workflow a step further and requiring developers to merge to an integration branch during development, before finally merging the completed story to master. This practice ferrets out integration issues even faster. Developers get to mash their in-progress stories together several times a day, rather than waiting until one has merged a completed story to master (which may happen only every 2-3 days) and the other has pulled code from master onto their story branch.

There are several variations of the integration branch workflow. Some teams merge work to integration while the story is in progress, then merge the story branch directly to master and close the branch as soon as the story (ie, the JIRA issue) is completed. Some teams merge the integration branch onto master every time it passes CI successfully (Bamboo's automated merging FTW!). Other teams work exclusively around the integration branch during their iteration and wait until the end when their product owner has blessed all the user stories there, before merging integration onto master.

Extra Credit

While not every team does it, even at Atlassian, many teams adopt the convention of using JIRA issue keys in their branch names.  It's an obvious way to keep track of which branch is implementing which issue, and is often coupled with a human-friendly descriptor - 'GHS-5517-issue-preview-pannel', for example.  Bamboo recently added features to support this even better: surfacing the branch's current build status right inside the JIRA issue, for one. 

While Bamboo supports story branching really, really well (you won't blame me for being biased, will you, Gentle Reader?), this workflow can be set up with just about any build system and just about any issue tracker.  Give it a try during your next iteration. And may you benefit from it as much as we have!