Release Feature Branching dramatically increases development costs and risk
Feature Branching is a version control practice in which developers commit their changes to a branch of a source code repository before merging into trunk at a later date. Popularised in the 1990s and 2000s by centralised Version Control Systems (VCS) such as ClearCase, Feature Branching has evolved over the years and is currently enjoying a resurgence in popularity thanks to Distributed Version Control System (DVCS) tools such as GitHub.
The traditional form of Feature Branching originally promoted by ClearCase et al might be called Release Feature Branching. The central branch known as trunk is considered a flawless representation of all previously released work, and new features for a particular release are developed on a long-lived branch. Developers commit changes to their branch, automated tests are executed, and testers manually verify the new features. Those features are then released into production from the branch, merged into trunk by the developers, and regression tested on trunk by the testers. The branch can then be earmarked for deletion and should only be used for production defect fixes.
For example, consider an IT organisation with a product stored in a monolithic codebase. Two new business projects 5.1 and 5.2 start concurrently for different customers, and development begins on corresponding feature branches. The 5.1 branch is broken early on in month 1, but 5.2 is unaffected and carries on regardless.
In month 2, another two projects named 5.3 and 5.4 begin. 5.3 is estimated to have a low impact but its branch is broken by a refactoring and work is rushed to meet the 5.3 deadline. Meanwhile the 5.4 branch is broken by a required architecture change and gradually stabilised.
In month 3, 5.3 is tested and released into production before being merged into trunk and regression tested. The 5.2 branch becomes broken so progress halts until it is fixed. The 5.1 branch is tested and released into production before the merge and regression testing of trunk + 5.3 + 5.1.
In month 4, 5.2 is tested and released into production but the subsequent merge and regression testing of trunk + 5.3 + 5.1 + 5.2 unexpectedly fails. While the 5.2 developers fix trunk 5.4 is tested and released, and once trunk is fixed the merge and regression testing of trunk + 5.3 + 5.1 + 5.2 + 5.4 is performed. Soon afterwards a critical defect is found in 5.4, so a 5.4.1 fix is also released.
At this point all 4 feature branches could theoretically be deleted, but the 5.1 customer requests another feature and refuses to upgrade due to the perceived risk. The dormant 5.1 branch is resurrected so 5.1.1 can be released into production and merged into trunk. While the 5.1 merge was trunk + 5.3 the 5.1.1 merge is trunk + 5.3 + 5.2 + 5.4.1, resulting in a more complex merge and extensive regression testing.
In this example 5.1, 5.2, 5.3, and 5.4 enjoyed between 1 and 3 months of uninterrupted development, and 5.4 was even released into production while trunk was broken. However, each period of isolated development created a feedback delay on trunk integration, and this was worsened by the localisation of design activities such as the 5.3 refactoring and 5.4 architectural change. This ensured merging and regression testing each branch would be a painful, time-consuming process that prevented new features from being worked on – except 5.1.1, which created an even more costly and risky integration into trunk.
This situation could have been alleviated by the 5.1, 5.2, 5.3, and/or 5.4 developers directly merging the changes on other branches into their own branch prior to their production release and merge into trunk. For instance, in month 4 the 5.4 developers might have merged the latest 5.1 changes, the latest 5.2 changes, and the final 5.3 changes into the 5.4 branch prior to release.
Martin Fowler refers to this process of directly merging between branches as Promiscuous Integration, and promiscuously integrating 5.1, 5.2, and 5.3 into 5.4 would certainly have reduced the complexity of the eventual trunk + 5.3 + 5.1 + 5.2 + 5.4 merge. However, newer 5.1 and 5.2 changes could still introduce complexity into that merge, and regression testing 5.4 on trunk would still be necessary.
The above example shows how Release Feature Branching inserts an enormously costly and risky integration phase into software delivery. Developer time must be spent managing and merging feature branches into trunk, and with each branch delaying feedback for prolonged periods a complex merge process per branch is inevitable. Tester time must be spent regression testing trunk, and although some merge tools can automatically handle syntactic merge conflicts there remains potential for Semantic Conflicts and subtle errors between features originating from different branches. Promiscuous Integration between branches can reduce merge complexity, but it requires even more developer time devoted to branch management and the need for regression testing on trunk is unchanged.
Since the mid 2000s, Release Feature Branching has declined in popularity thanks to the rise of Continuous Integration and the argument popularised by Dan Bodart that “Feature Branching is a poor man’s modular architecture” – integrating small changesets into trunk on at least a daily basis minimises merge costs and eliminates regression testing, and the greater emphasis on communication facilitates a more flexible architecture that allows features to be controlled at runtime via configuration rather than at build time via version control. Organisations that still rely upon long-lived feature branches today have a significant business problem to overcome. Branching, merging, and regression testing are all non-value adding activities that reduce available time for feature development, and as branches discourage collaboration there will be gradual deterioration in codebase quality making it harder to add new features. This is why it is important to heed the advice of Dave Farley and Jez Humble that “you should never use long-lived, infrequently merged branches as the preferred means of managing the complexity of a large project“.