Automated Pipelines Using CircleCI
Those new to using CircleCI can use this tutorial to create a basic automated pipeline with the tool.
Join the DZone community and get the full member experience.Join For Free
We won't go into the details of the full DevOps process (this might be for another post), but we will start something simple: developing an automated pipeline for deployments. Before going into an explanation of CircleCI, we have to review the basics of continuous integration and continuous deployment.
Continuous Integration and Continuous Deployment
Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.
Continuous deployment (CD) is a strategy for software releases wherein any code commit that passes the automated testing phase is automatically released into the production environment, making changes that are visible to the software’s users.
Here are a few points to be noted,
- For each and every check-in into a version control system, this pipeline would be triggered.
- The confidence of the build would increase as it goes through each stage.
- We can have an approval process for higher-level environments and have automated deployment to a development environment (remember this is an example; your requirements might vary).
Now let's take a look at setting up this pipeline with CircleCI.
CircleCI is a hosted CI server which integrates with Github very well.
Setting Up The Project
When you sign into CircleCI using a Github account, it will ask permission to read your projects. If you give permission, you can go to CircleCI and get all your projects listed there. Click on "Set Up Project" and proceed.
CircleCI nees us to create the configuration file at this path, PROJECT_ROOT/.circleci/config.yml. So create that file at the root of your project.
If you want to test the CircleCI configuration in local you can use these commands,
circleci config validate: Validates the yaml content of the CircleCI configuration
circleci local execute — job linting: Executes the job in local
If you want to setup the CircleCI CLI in local, please read this post.
In the above sample pipeline diagram, each and every box is a job. It runs the actual set of steps given inside a machine. The machine that is running the steps is called an executor. There are multiple executors (Docker, Shell, VM), but we will use Docker Executor and create a "Hello World" example.
version: 2 jobs: build: docker: - image: busybox:latest steps: - run: name: Hello World command: echo "Hello World"
Here we declared that
- We are going to use CircleCI version 2
- Created one job called build. If we don’t have workflows (which we will talk about next) then that default job name should be build
- We used Docker executor with busybox as the image
- The list of steps to run inside the Busybox container
But this example is not sufficient to build a complete pipeline. To create multiple jobs and link them together, we will need to create workflows.
version: 2 jobs: unit_test: docker: - image: busybox:latest steps: - run: name: Unit Test command: echo "Going to run Unit Test" linting: docker: - image: busybox:latest steps: - run: name: Lint Test command: echo "Going to run Lint Test" workflows: version: 2 sample_pipeline: jobs: - unit_test - linting
We created the workflow named sample_pipeline and gave 2 jobs to it.
- These two jobs, unit_test and linting would be running in parallel and one doesn’t depend on the output of other.
- Also each job would gets its own executor. In our case, both the jobs use Docker Executor.
Here we will create multiple jobs with dependencies between them,
version: 2 jobs: unit_test: docker: - image: busybox:latest steps: - run: name: Unit Test command: echo "Going to run Unit Test" linting: docker: - image: busybox:latest steps: - run: name: Lint Test command: echo "Going to run Lint Test" build_artifact: docker: - image: busybox:latest steps: - run: name: Build Artifact command: echo "Going to run the build artifact" deploy_to_dev: docker: - image: busybox:latest steps: - run: name: Deploy to Dev command: echo "Going to deploy to dev" deploy_to_qa: docker: - image: busybox:latest steps: - run: name: Deploy to QA command: echo "Going to deploy to QA" - run: name: Step no 2 command: echo "Going to deploy to QA again" workflows: version: 2 sample_pipeline: jobs: - linting - unit_test - build_artifact: requires: - unit_test - linting - deploy_to_dev: requires: - build_artifact - deploy_to_qa: requires: - deploy_to_dev
There are a few points to note here:
unit_testjobs would still run parallel
- Fan In: the
build_artifactjob would be triggered only when both
lintingare successful, so both
unit_testfan in to the
- A job can have multiple steps, as given in
deploy_to_qa. These steps are always executed in sequential order and always uses the same container.
The only missing part here is the approval process. If we don’t have that, then the whole pipeline would be running for each and every check-in, which we don’t want to have (again, this depends).
... workflows: version: 2 sample_pipeline: jobs: - linting - unit_test - build_artifact: requires: - unit_test - linting - deploy_to_dev: requires: - build_artifact - hold_for_approval: type: approval requires: - deploy_to_dev - deploy_to_qa: requires: - hold_for_approval
For readability, I have pasted only the workflows and left out the jobs part. Again, you can check the entire configuration here. Some points to be noted,
hold_for_approvalis a new job with type marked as approval.
deploy_to_qawould be triggered only when
Once everything is completed, this would be the pipeline output.
This post was originally published in Francium Tech Blogs.
Published at DZone with permission of Prasanna Venkataraman. See the original article here.
Opinions expressed by DZone contributors are their own.