The Top 7 Pipeline Design Patterns for Continuous Delivery

Business agility is built on code agility. For modern digital products and services, the ability to release on demand quickly and safely is a real competitive business advantage.

Since 2004 we have designed, built, and deployed code pipelines to automate applications and infrastructure. We’re sharing seven pipeline design patterns that we’ve learned improve speed, agility, and quality while increasing autonomy, transparency, and maintainability.

Continuous Delivery

Continuous Delivery is “the ability to get changes of all types into the hands of users, safely and quickly in a sustainable way.” If you think about Continuous Delivery on an Agile vs. Effort matrix, it falls squarely between Continuous Integration and Continuous Deployment. Often these are collectively referred to as CI/CD.

In the 2019 State of the DevOps Report, over 31,000 respondents reported the effectiveness of their development and delivery processes. The results between the elite and low performers is staggering. The elite performs had 200x more deployments and 100x faster speed to deployment while also being 2,600x faster to recover from incidents and 7x less likely to roll back releases.

This research shows that for elite performing organizations, speed and stability are not opposites! You can have both – in fact you need both – to gain real competitive advantages for your digital products and services.

Code Pipelines

Code pipelines are the primary technical artifacts of continuous delivery. Modern-day pipelines transform application and infrastructure source code into versioned packages deployable to any environment. By automating all the mundane tasks to build and deploy systems, teams are free to focus on value-added capabilities.

While code pipelines have been around for nearly 20 years – CruiseControl, one of our early favorites, was first released in 2001 – they have evolved quite significantly over the years and really come into their own in the last few years.

Based on our work and observations from our customers, we have identified 7 pipeline design patterns that we see in many modern tech organizations.

Pipeline Design Patterns

Pipeline Design Pattern #1: Pipelines as Code

Pipeline logic is codified, stored alongside application or infrastructure code and utilizes containerized runners.

State of the Art:

• No GUI setup steps! Pipeline logic is managed like any other application code and subject to the same branching strategies and review processes.
• Pipeline execution on containers allows your CI/CD platform to support many different workloads, while each workload can have its own build environment to meet its unique needs.
• Container image provenance uses trusted Docker images for build environments.
• CI runner configuration is automated, identical, and hands-free. CI runners can scale to meet demand and can be on standby during core hours to minimize delays.
• Secrets are stored outside of the pipeline and their output is masked, leading to higher security.

Pipeline Design Pattern #2: Externalize Logic into Reusable Libraries

Reusable libraries contain common pipeline logic that is referenceable from pipeline code and independently developed and tested.

State of the Art:

• Treat pipeline libraries as any other releasable software. They have their own repos, pipelines, and are unit tested with good release notes.
• Pipelines call external tasks whenever possible using language-specific tools like Make, Rake, npm, Maven or others to simplify the pipeline and keeps the local and CI workflow identical.
• Libraries are discoverable and have good documentation.

Pipeline Design Pattern #3: Separate Build and Deploy Pipelines

Build and deploy pipelines should be logically separated, independently runnable and triggered by automated or manual events

State of the Art:

• Build once, deploy many. Focus on the first build. It becomes an artifact that you can deploy many times.
• Be environmentally agnostic. Without an environment-specific packages, and externalized environment-specific properties, the same build can run in any environment.
• Package it all together. Everything – All source, including infrastructure code, should be combined, becoming a versioned package.

Pipeline Design Pattern #4: Trigger the Right Pipeline

Branch commits, pull requests, and merges to the mainline can all trigger different pipeline behavior, optimized to the team’s way of working.

State of the Art:

• Pushing a commit to an open pull request builds an Ephemeral Environment for testing.
• Merges to the mainline are deployed to a non-production or demo environment displaying the latest integrated code
• Pushing a new tag stages a production release.

Pipeline Design Pattern #5: Fast Team Feedback

Every commit automatically triggers the right pipeline, with build pipelines especially optimized for speed and quick reporting of any issues.

State of the Art:

• Build pipelines use parallelization for non-interdependent jobs to increase speed.
• Fast build pipelines only run the jobs that are necessary in a few minutes.
• Each successful run produces a versioned package and static analysis results.
• With omnichannel notifications, you can enable team notifications on pull request status in dashboards, chat channels, email, and other mediums.

Pipeline Design Pattern #6: Stable Internal Releases

Only versioned packages produced by the build pipeline are deployed and these deployments are triggered by humans or automated events.

State of the Art:

• Each code branch gets a complete ephemeral environment named for the branch that can easily be created or destroyed.
• Each engineer can stand up and delete ephemeral environments at any time.
• CI runners use cloud-native IAM capabilities with temporary permissions so they can assume roles and acquire the right permissions to complete their work.

Pipeline Design Pattern #7: Buttoned Up Product Releases

Deploy tagged releases to production and automate the paperwork but leave a paper trail.

State of the Art:

• Codified release gates and standardized release processes enables teams to release on demand.
• Automated releases leave a transparent paper trail that’s auditable for governance and quality
• Release gates can invoke external API’s and use the responses to decide whether to proceed with the release or halt.

Challenges

These are the seven pipeline design patterns we’re increasingly seeing and using in our client engagements. While these present huge leaps forward in terms of speed and stability, code pipelines are not without their challenges.

Security is the biggest challenge we see, due to the complexity that comes with automating what has traditionally been human-centric processes. Pipeline complexity, team adoption, modernizing the change-advisory-board-centric culture, and automating databases are the other big challenges to work through. But it’s possible, we do this every day.

Summary

Business agility is built on code agility. For modern digital products and services, the ability to release on demand quickly and safely is a real competitive business advantage. Code pipelines, and these seven design patterns in particular, can help your organization make giant leaps forward in speed and stability and have your teams performing at the elite level.