DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. DevOps and CI/CD
  4. Automated Pipelines Using CircleCI

Automated Pipelines Using CircleCI

Those new to using CircleCI can use this tutorial to create a basic automated pipeline with the tool.

Prasanna Venkataraman user avatar by
Prasanna Venkataraman
·
May. 24, 19 · Tutorial
Like (1)
Save
Tweet
Share
7.38K Views

Join the DZone community and get the full member experience.

Join For Free

This topic was taken as a session as part of the #ThursdayDevops series we have here at Francium Tech. You can find the information about sessions in this Github repo.

We won't go into the details of the full DevOps process (this might be for another post), but we will start something simple: developing an automated pipeline for deployments. Before going into an explanation of CircleCI, we have to review the basics of continuous integration and continuous deployment.

Continuous Integration and Continuous Deployment

Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

Continuous deployment (CD) is a strategy for software releases wherein any code commit that passes the automated testing phase is automatically released into the production environment, making changes that are visible to the software’s users.

This itself is a vast topic, you can read more about it here and here. Here is a sample pipeline setup for an automated build,


Here are a few points to be noted,

  • For each and every check-in into a version control system, this pipeline would be triggered.
  • The confidence of the build would increase as it goes through each stage.
  • We can have an approval process for higher-level environments and have automated deployment to a development environment (remember this is an example; your requirements might vary).

Now let's take a look at setting up this pipeline with CircleCI.

CircleCI

CircleCI is a hosted CI server which integrates with Github very well.

Setting Up The Project

When you sign into CircleCI using a Github account, it will ask permission to read your projects. If you give permission, you can go to CircleCI and get all your projects listed there. Click on "Set Up Project" and proceed.

Circle CI project setup

Helper Utilities

CircleCI nees us to create the configuration file at this path, PROJECT_ROOT/.circleci/config.yml. So create that file at the root of your project.

If you want to test the CircleCI configuration in local you can use these commands,

  •  circleci config validate: Validates the yaml content of the CircleCI configuration

  •  circleci local execute — job linting: Executes the job in local

If you want to setup the CircleCI CLI in local, please read this post.

Jobs

In the above sample pipeline diagram, each and every box is a job. It runs the actual set of steps given inside a machine. The machine that is running the steps is called an executor. There are multiple executors (Docker, Shell, VM), but we will use Docker Executor and create a "Hello World" example.

version: 2
jobs:
        build:
                docker:
                - image: busybox:latest
                steps:
                - run:
                        name: Hello World
                        command: echo "Hello World"


Here we declared that

  • We are going to use CircleCI version 2
  • Created one job called build. If we don’t have workflows (which we will talk about next) then that default job name should be build
  • We used Docker executor with busybox as the image
  • The list of steps to run inside the Busybox container

Workflows

But this example is not sufficient to build a complete pipeline. To create multiple jobs and link them together, we will need to create workflows.

version: 2
jobs:
        unit_test:
                docker:
                - image: busybox:latest
                steps:
                - run:
                        name: Unit Test
                        command: echo "Going to run Unit Test"
        linting:
                docker:
                - image: busybox:latest
                steps:
                - run:
                        name: Lint Test
                        command: echo "Going to run Lint Test"

workflows:
        version: 2
        sample_pipeline:
                jobs:
                        - unit_test
                        - linting


We created the workflow named sample_pipeline and gave 2 jobs to it.

  • These two jobs, unit_test and linting would be running in parallel and one doesn’t depend on the output of other.
  • Also each job would gets its own executor. In our case, both the jobs use Docker Executor.

Pipeline Setup

Here we will create multiple jobs with dependencies between them,


version: 2
jobs:
        unit_test:
                docker:
                - image: busybox:latest
                steps:
                - run:
                        name: Unit Test
                        command: echo "Going to run Unit Test"
        linting:
                docker:
                - image: busybox:latest
                steps:
                - run:
                        name: Lint Test
                        command: echo "Going to run Lint Test"
        build_artifact:
                docker:
                - image: busybox:latest
                steps:
                - run:
                        name: Build Artifact
                        command: echo "Going to run the build artifact"

        deploy_to_dev:
                docker:
                - image: busybox:latest
                steps:
                - run:
                        name: Deploy to Dev
                        command: echo "Going to deploy to dev"
        deploy_to_qa:
                docker:
                - image: busybox:latest
                steps:
                - run:
                        name: Deploy to QA
                        command: echo "Going to deploy to QA"
                - run:
                        name: Step no 2
                        command: echo "Going to deploy to QA again"
workflows:
        version: 2
        sample_pipeline:
                jobs:
                        - linting
                        - unit_test                                
                        - build_artifact:
                                requires:
                                - unit_test
                                - linting
                        - deploy_to_dev:
                                requires:
                                - build_artifact
                        - deploy_to_qa:
                                requires:
                                - deploy_to_dev


There are a few points to note here:

  • Both linting and unit_test jobs would still run parallel
  • Fan In: the build_artifact job would be triggered only when both unit_testandlinting are successful, so both linting & unit_test fan in to thebuild_artifact .
  • A job can have multiple steps, as given in  deploy_to_qa. These steps are always executed in sequential order and always uses the same container.

Approval Process

The only missing part here is the approval process. If we don’t have that, then the whole pipeline would be running for each and every check-in, which we don’t want to have (again, this depends).

...
workflows:
        version: 2
        sample_pipeline:
                jobs:
                        - linting
                        - unit_test
                        - build_artifact:
                                requires:
                                - unit_test
                                - linting
                        - deploy_to_dev:
                                requires:
                                - build_artifact
                        - hold_for_approval:
                                type: approval
                                requires:
                                - deploy_to_dev
                        - deploy_to_qa:
                                requires:
                                - hold_for_approval


For readability, I have pasted only the workflows and left out the jobs part. Again, you can check the entire configuration here. Some points to be noted,

  • hold_for_approval is a new job with type marked as approval.
  • deploy_to_qa would be triggered only when hold_for_approval is done

Once everything is completed, this would be the pipeline output.


This post was originally published in Francium Tech Blogs.

Pipeline (software) Continuous Integration/Deployment career

Published at DZone with permission of Prasanna Venkataraman. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Core Machine Learning Metrics
  • DevOps Roadmap for 2022
  • Silver Bullet or False Panacea? 3 Questions for Data Contracts
  • Multi-Cloud Database Deep Dive

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: