Automated Pipelines Using CircleCI
Those new to using CircleCI can use this tutorial to create a basic automated pipeline with the tool.
Join the DZone community and get the full member experience.
Join For FreeThis topic was taken as a session as part of the #ThursdayDevops series we have here at Francium Tech. You can find the information about sessions in this Github repo.
We won't go into the details of the full DevOps process (this might be for another post), but we will start something simple: developing an automated pipeline for deployments. Before going into an explanation of CircleCI, we have to review the basics of continuous integration and continuous deployment.
Continuous Integration and Continuous Deployment
Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.
Continuous deployment (CD) is a strategy for software releases wherein any code commit that passes the automated testing phase is automatically released into the production environment, making changes that are visible to the software’s users.
This itself is a vast topic, you can read more about it here and here. Here is a sample pipeline setup for an automated build,
Here are a few points to be noted,
- For each and every check-in into a version control system, this pipeline would be triggered.
- The confidence of the build would increase as it goes through each stage.
- We can have an approval process for higher-level environments and have automated deployment to a development environment (remember this is an example; your requirements might vary).
Now let's take a look at setting up this pipeline with CircleCI.
CircleCI
CircleCI is a hosted CI server which integrates with Github very well.
Setting Up The Project
When you sign into CircleCI using a Github account, it will ask permission to read your projects. If you give permission, you can go to CircleCI and get all your projects listed there. Click on "Set Up Project" and proceed.

Helper Utilities
CircleCI nees us to create the configuration file at this path, PROJECT_ROOT/.circleci/config.yml. So create that file at the root of your project.
If you want to test the CircleCI configuration in local you can use these commands,
circleci config validate
: Validates the yaml content of the CircleCI configurationcircleci local execute — job linting
: Executes the job in local
If you want to setup the CircleCI CLI in local, please read this post.
Jobs
In the above sample pipeline diagram, each and every box is a job. It runs the actual set of steps given inside a machine. The machine that is running the steps is called an executor. There are multiple executors (Docker, Shell, VM), but we will use Docker Executor and create a "Hello World" example.
version: 2
jobs:
build:
docker:
- image: busybox:latest
steps:
- run:
name: Hello World
command: echo "Hello World"
Here we declared that
- We are going to use CircleCI version 2
- Created one job called build. If we don’t have workflows (which we will talk about next) then that default job name should be build
- We used Docker executor with busybox as the image
- The list of steps to run inside the Busybox container
Workflows
But this example is not sufficient to build a complete pipeline. To create multiple jobs and link them together, we will need to create workflows.
version: 2
jobs:
unit_test:
docker:
- image: busybox:latest
steps:
- run:
name: Unit Test
command: echo "Going to run Unit Test"
linting:
docker:
- image: busybox:latest
steps:
- run:
name: Lint Test
command: echo "Going to run Lint Test"
workflows:
version: 2
sample_pipeline:
jobs:
- unit_test
- linting
We created the workflow named sample_pipeline and gave 2 jobs to it.
- These two jobs, unit_test and linting would be running in parallel and one doesn’t depend on the output of other.
- Also each job would gets its own executor. In our case, both the jobs use Docker Executor.
Pipeline Setup
Here we will create multiple jobs with dependencies between them,
version: 2
jobs:
unit_test:
docker:
- image: busybox:latest
steps:
- run:
name: Unit Test
command: echo "Going to run Unit Test"
linting:
docker:
- image: busybox:latest
steps:
- run:
name: Lint Test
command: echo "Going to run Lint Test"
build_artifact:
docker:
- image: busybox:latest
steps:
- run:
name: Build Artifact
command: echo "Going to run the build artifact"
deploy_to_dev:
docker:
- image: busybox:latest
steps:
- run:
name: Deploy to Dev
command: echo "Going to deploy to dev"
deploy_to_qa:
docker:
- image: busybox:latest
steps:
- run:
name: Deploy to QA
command: echo "Going to deploy to QA"
- run:
name: Step no 2
command: echo "Going to deploy to QA again"
workflows:
version: 2
sample_pipeline:
jobs:
- linting
- unit_test
- build_artifact:
requires:
- unit_test
- linting
- deploy_to_dev:
requires:
- build_artifact
- deploy_to_qa:
requires:
- deploy_to_dev
There are a few points to note here:
- Both
linting
andunit_test
jobs would still run parallel - Fan In: the
build_artifact
job would be triggered only when bothunit_test
andlinting
are successful, so bothlinting
&unit_test
fan in to thebuild_artifact
. - A job can have multiple steps, as given in
deploy_to_qa
. These steps are always executed in sequential order and always uses the same container.
Approval Process
The only missing part here is the approval process. If we don’t have that, then the whole pipeline would be running for each and every check-in, which we don’t want to have (again, this depends).
...
workflows:
version: 2
sample_pipeline:
jobs:
- linting
- unit_test
- build_artifact:
requires:
- unit_test
- linting
- deploy_to_dev:
requires:
- build_artifact
- hold_for_approval:
type: approval
requires:
- deploy_to_dev
- deploy_to_qa:
requires:
- hold_for_approval
For readability, I have pasted only the workflows and left out the jobs part. Again, you can check the entire configuration here. Some points to be noted,
hold_for_approval
is a new job with type marked as approval.deploy_to_qa
would be triggered only whenhold_for_approval
is done
Once everything is completed, this would be the pipeline output.

This post was originally published in Francium Tech Blogs.
Published at DZone with permission of Prasanna Venkataraman. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments