Is Continuous Delivery just Continuous Integration done right?
A couple days ago, Steve Smith started an interested conversation on Twitter by pointing out that using a normal CI tool to manage your CD pipeline is a mistake.
— Steve Smith (@AgileSteveSmith) April 2, 2014
I agree with Steve 100%. But in a moment of snark, I took the conversation into a place where we found disagreement:
— Steve Smith (@AgileSteveSmith) April 3, 2014
So, why would I suggest something that seems contrary to our agreed upon position that using CI tools for CD is an anti-pattern? Well, it gets to definitions of things, subtle distinctions and the like. If you’re looking for simple lessons, stop here and don’t use a build tool for deployments. If you want to understand why I think CI and CD done correctly are almost the same thing, follow me down the rabbit hole.
The Martin Fowler definition of CI:
Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.
And Fowler’s definition of CD:
Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time.
I’m on the record suggesting that Fowler is generally wrong on his emphasis on “builds.” The build does happen to provide an early validation of functionality via the “does it compile” test, and a useful packaging of code that is part of the same service. Beyond those tasks, it is incidental to the verification of the change by testing. So let’s drop the focus on build and use these definitions:
CI: Continuous integration is a software delivery practice where members of a team integrate their work frequently. Each integration is tested so that the team is has rapid feedback of integration errors, and other issues causing the software to be worse.
CD: Continuous Delivery is a software delivery discipline where you develop and test your software in such a way that you can release at any time the business desires, and actually do release frequently.
Continuous Integration is fundamentally about making sure that the changes from one developer work with the changes from another developer. When Fowler first write the “canonical” CI article in Sept, 2000. The primary place integration happened was in a common code line in source control. Frequent builds with unit tests were a pretty good validation that integration went ok. As such, “Continuous Integration Tool” (and CI itself) became seen by the community/market as basically “Continuous Build.”
Today, meaningful integration happens less in source control and more frequently in a runtime environment. By the time you log in to your account at your bank and check your balance, you have likely interacted with dozens or hundreds of services supporting a single, simple user story. And so, if you want to quickly verify that the work of two developers is not conflicting, you need to run API tests, as well as some functional tests scenarios. That means bringing the various services into an integration test environment, and validating runtime functionality at a pace similar to the classic CI build loop.
Did this change hurt performance? Performance test it. Security? Run the static and dynamic security tests? Did it hurt our revenue per visitor? A/B test it in production.
Yes, that’s right, deployment to production as part of CI testing. If we think about how we are going to check for “integration errors” or general degradation of the application as a result of frequently integration code, we need to deploy our software into a number of environments (potentially including production) and test / monitor it in runtime. CI done right, is continuous delivery.
Meanwhile, CD is basically just striving for continuously shippable code. Some of that suggests fancy techniques like feature flags hiding half done but integrated features. The core of CD is just rapidly testing new changes to make sure the code base is still shippable. That’s just continuous integration as described above.
CI is about rapid testing. CD is about rapid delivery of value. These practices care about very different things. Properly implemented, they look very similar. We saw the same pattern with build management and continuous integration back in the 2001-2003 time period. Tools like CruiseControl were built strictly for rapid feedback (CI). Tools like Anthill were built for correct execution of traceable builds (build management). 80% of the implementation was the same. Over a few years, build management and CI tools stole one another’s features and merged into the same class of tool.
So, should I use CI tools for CD then?
No. No. No. No.
Most “CI” tools are at their heart continuous build (with unit test) tools. Build optimized tools and Deployment optimized tools are very different. Stealing a handful of features from the other class of tool, as we see with CI tools bolting on pipelines, probably won’t work. Too many fundamental assumptions are wrong.
Regardless of how clever I may be in showing that CI done right is very similar to CD, the common understanding of CI aligns with the thinking from circa 2000 where interesting integration was validated by builds. That probably isn’t going to change. Given that, using a different term to describe the richer scenario is probably wise. Long live “continuous delivery.”