Continuous Delivery’s Trouble with Web Services
Typical Continuous Delivery, where builds are promoted along a pipeline or lifecycle is broken for complex architectures – especially web service based applications. I love these build promotion approaches and have cheered them on for six years, but they use a very imperfect model.
In the diagram below, we look at a typical build deployment pipeline including a build, a variety of testing environments, approvals, and eventual release to the live production environments. The three components (Services 1,2,3) are all set to use this model independently.
There are several problems when using this model:
- The web services are interdependent. A change in an API for #1 may break #2 unless an update for it is also released.
- Because things are released together, the “sign-off” rarely applies to the release of a single version of a single component. Instead, a collection of items is approved.
- The pipeline always starts with a build. While appropriate for most web services, releases often include components that are not built: incremental static content moves, database updates or report migrations.
- The build pipeline is too rigidly ordered (a complaint I’ve made before).
The book also suggests an “Integration Pipeline” in Chapter 13. In this approach each major component has a short testing pipeline and is marked as good. A new build containing all the latest good components is created and moves through its own pipeline towards production release. The authors bend over backwards encouraging you to avoid this practice if you can.
I understand why all too well. The “build” that contains the latest good versions of each of the components is hard to work with. Sometimes we only want to release half the components. Sometimes, the “latest good” of one component is incompatible with the “latest good” of another and we would rather release a slightly older version of that component. Only with exceptional discipline can we eliminate the numerous failure modes of assembling all the “latest good” and people are not good at discipline.
Using a build pipeline to model release orchestration and planning activities is simply a bad fit. It feels like wearing shoes a size too small. You can squeeze into them and they protect the soles of your feet, but expect pain.
What we want to do instead is to decouple our deployment pipeline or delivery chain from the concept of “a build”. Instead, we should focus on the idea of a Release Set or Snapshot which is a lightweight deployable unit for these larger systems.
Snapshots are similar to Integration Pipeline builds in that they reference versions of related components. But rather than retrieve and assemble those related components into a final application in as part of a super build, we instead, use a release set to logically collect the component versions. The Release Set or Snapshot is not “built” so much as “declared” and is analogous to the classic Release Manager’s spreadsheet that details the versions of each component that should be tested and released together. The Release Set also provides a natural granularity for managing approvals.
We also need to reject the notion that a Release Set will contain exactly one version of each component. For instance, in the Report migration example above, there might be a number of different new reports that have been independently developed and tested. If the component is naturally promoted in an incremental fashion, we include multiple, ordered versions in the Release Set.
Last week I worked with a team that was trying to figure out how to better update a complex CRM application. As an example, the team shared a recent release plan detailing updates to each of several dozen components. Modeling that in a build pipeline would be a nightmare. With release snapshots, we have a promising way forward.
I’ll be talking a lot more about the limitations of traditional CD, approaches for accomadating those limitations, and alternate models like Release Sets in our upcoming webcast, “Next Gen Continuous Delivery.” Please join us for that event.