CI/CD Overview
The value of our software is not only found in the code we write, but also in how we package and deliver that code. Beyond the code we write, we must also do the following to meet the needs of our customers and drive revenue:
- Create executables
- Test that the executables meet their specifications
- Deliver the executables
Since the inception of software development, this has typically been a manual process in which engineers create executables, the Quality Assurance (QA) team manually runs through a suite of tests and signs off on them, and operations staff manually deploy the certified executables into a production environment. In the last two decades though, operations and release engineering has progressed to the point where nearly all steps in the delivery process can be automated.
The series of steps that brings software from code to its execution in production is inherently driven by business needs. For example, some applications may require that acceptance and performance tests be run before the application can be deployed, while other applications may only require unit and integration tests. Likewise, some applications may be deployed to a single production environment, while others may package up an executable that can be downloaded by countless customers. Although the specific steps will vary by our business needs, the general principle remains the same: executing steps in series and in parallel.
When executed manually, this is an arduous, costly process and becomes increasingly burdensome as more steps are added.
Continuous Integration
The first step in the march toward automation was Continuous Integration (CI). Prior to CI, developers would work on separate pieces of an application and test them in isolation. Once enough code was ready, the development team would combine their parts and test them together in a process that came to be known as Big Bang Integration. In most cases, integration never worked as expected and would take days or weeks to resolve any problems that occurred. This costly process would repeat every few weeks until all the desired features were ready, and when done at the last minute, it could prove disastrous.
To improve this process, developers started automating their unit and integration tests, running them at least once a day or, if possible, after each check-in to the repository. This approach ensured that integration was done dozens or even hundreds of times a day and that tiny segments of the application — rather than large, monolithic chunks — were integrated. This CI process drastically improved both turnaround time and application quality, setting the groundwork for the next logical step in the automation process: Continuous Delivery (CD).
Continuous Delivery
While CI focuses on the integration of code, CD automates the entire delivery process, from check-in to deployment. The core concept of CD is a pipeline, which represents a set of ordered stages — some are executed in series and others are executed in parallel. An example pipeline is illustrated below:
In this case, the CD pipeline starts with a commit to the repository, which then initiates a build. Once the build is complete and an executable is created, the executable is unit tested. If all unit tests pass, the executable is integration tested; if all integration tests pass, the acceptance tests are run. And once all acceptance tests pass, the pipeline runs three stages in parallel within a production environment:
- User Acceptance Testing (UAT) – a set of manual tests, such as UI tests or other acceptance tests, that require human judgment to determine if they pass. Since completion of this stage requires manual sign-off, the progression of the pipeline is said to be gated until this stage is complete.
- Capacity Testing – performance tests that exercise if the application meets its timing and capacity specifications in a production environment.
- Staging – the executables and its accompanying configuration are staged and prepared for deployment.
If all three of these stages pass — in the case of the manual UATs, when a tester signs off that the tests have passed — the staged executables and configuration are deployed to the production environment (or moved to a public-facing server so that customers can download the application). While some stages can be completed manually by pressing a button in the pipeline, it is best to automate as many, if not all, stages as possible. A fully automated pipeline allows developers to check in code to a repository and, if all tests pass, ensure that their changes are pushed to production in a consistent and repeatable manner without any human interaction.
The concepts of CI/CD are crucial to the advancement of software development, and their business and practical benefits are reaped when we integrate them into our repositories. While there are countless repositories to choose from, one has become the most popular option by far: GitHub.
{{ parent.title || parent.header.title}}
{{ parent.tldr }}
{{ parent.linkDescription }}
{{ parent.urlSource.name }}