{{announcement.body}}
{{announcement.title}}

Everything As A Code: Cloud Formation (ECS Fargate) Template Through Jenkins

DZone 's Guide to

Everything As A Code: Cloud Formation (ECS Fargate) Template Through Jenkins

This tutorial demonstrates how, with just the Jenkins Master and codbase, you can create a pipeline code for a customized Dockerfile.

· DevOps Zone ·
Free Resource

Introduction

This article is an example of “Everything as a code” to create an ECS Fargate Stack in AWS using the Cloud Formation template through Jenkins.

Only Jenkins Master and codebase is required to check this flow. Check out the Github URL mentioned below to refer to the code.

You may also enjoy: The Best of Both Worlds With AWS Fargate

Jenkins Configuration

There is some Jenkins master node configuration required to trigger the pipeline. 

Docker Installation

Configure the Docker Installation in Jenkins > Manage Jenkins > Global Tool Configuration > Docker.

Docker Installation configuration in Jenkins Master

Fig 1: Docker Installation configuration in Jenkins Master

AWS Credentials

Need to store the AWS Credentials on Jenkins > Credentials > System > Global Credentials > Add Credentials

Fig 2: AWS Credentials configuration in Jenkins

Fig 2: AWS Credentials configuration in Jenkins

Then it will look like this under Jenkins > Credentials:

Be Familiar with Files and Folders 

In the code repository, there are some files and folder as below.

Jenkinsfile parameters and templates

  1. Templates folder — contains the YAML file which has Cloud Formation code to create the ECS Fargate.
  2. Parameters folder — contains the JSON file and it contains all the parameters which are required Cloud Formation YAML
  3. Dockerfile — A customized Dockerfile is created which has capabilities of AWS CLI and Cloud Formation Linting; no need to install these on Jenkins Master specifically.
  4. Jenkinsfile — The  Jenkins Pipeline which will trigger all the stages like “Poll SCM,” “Validating CF template and JSON,” and “ Cloud Formation Stack creation.”

Approach 

There are so many approaches to achieve the same task. The first few of the CLI (AWSCLI and CFN-LINT) need to be installed on Jenkins Master or many Jenkins Plugins are available to trigger the ECS Fargate parameters.

Here, an approach is taken where code will take off everything and Jenkins Pipeline is written completely based on code.

A modular approach is taken for Cloud Formation code where templates and parameters are kept in the separate directory, so it will be easy to main the CF code better.

A customized Dockerfile will take care of the AWS CLI and CFN-LINT; no need to install these on Jenkins Master.

Jenkinsfile has 3 stages: 

  1. Fetching code from the remote Git repository.
  2. Validating the CF template and JSON file before creating the ECS Fargate stack. The CF template will be validated using CFN-LINT running inside the container. However, the Python “json.tool” module is used to validate the JSON file. If there is a problem with the CF template and JSON file validation, then this stage will be failed.
  3. Here, ECS Fargate stack creation will be triggered, the ID will be successful, then it will return a stack ID.

Dockerfile code snapshot:

Dockerfile
 




xxxxxxxxxx
1
15


 
1
FROM alpine:3.8
2
# Default evironment variables.
3
ENV AWS_DEFAULT_REGION us-east-1
4
ENV AWS_ACCESS_KEY_ID aws_key_id
5
ENV AWS_SECRET_ACCESS_KEY aws_Secret_key
6
COPY templates/New_ECS_Fargate_Para.yml .
7
COPY parameters/parameters.json .
8
RUN apk add -q --update \
9
    && apk add -q \
10
            python py-pip py-setuptools \
11
            ca-certificates \
12
    && pip install --upgrade pip \
13
    && pip install awscli \
14
    && pip install cfn-lint \
15
    && rm -rf /var/cache/apk/*


Jenkinsfile code snapshot:

Groovy
 




xxxxxxxxxx
1
32


 
1
node() {
2
        stage("POLL SCM") {
3
                git credentialsId: 'gitlab', url: 'https://github.com/goyalvickey78/CF_ECS_FARGATE_Jenkins.git'
4
        }
5
        stage("CF Lintingi and Json Validation"){
6
            withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'ACCESS_ID', credentialsId: 'ECS_FARGATE', secretKeyVariable: 'ACCESS_KEY']]) {
7
                docker.withTool('myDocker') {
8
                    //Buidling a docker images based on the dockerfile
9
                    sh "docker build -t awscli ."
10
                    //Validating the CF template with cfn-lint and container will be created & validate
11
                    sh "docker run --rm awscli cfn-lint New_ECS_Fargate_Para.yml"
12
                    //Validating the Parameter JSON file with Pyton json.tool module
13
                    sh "python -m json.tool parameters.json"
14
                    sh "docker rmi awscli"
15
                }
16
            }
17
        }
18
       stage("ECS Fargate CF Trigger") {
19
            // Using the AWS credentials which are stored in Jenkins
20
            withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'ACCESS_ID', credentialsId: 'ECS_FARGATE', secretKeyVariable: 'ACCESS_KEY']]) {
21
                //Using Docker tool configured in Jenkins
22
                docker.withTool('myDocker') {
23
                        sh "docker build -t awscli ."
24
                        //Cloud Formation ECS Fargate Stack will be triggered with AWS CLI
25
                        sh "docker run -e \"AWS_ACCESS_KEY_ID=${ACCESS_ID}\" -e \"AWS_SECRET_ACCESS_KEY=${ACCESS_KEY}\" --rm awscli aws cloudformation create-stack --stack-name jfrogexample1 --template-body file://New_ECS_Fargate_Para.yml --parameters file://parameters.json"
26
                        //It will show all EC2 instsnces in AWS account
27
                        sh "docker run -e \"AWS_ACCESS_KEY_ID=${ACCESS_ID}\" -e \"AWS_SECRET_ACCESS_KEY=${ACCESS_KEY}\" --rm awscli aws ec2 describe-instances --output table"
28
                        sh "docker rmi awscli"
29
               }
30
            }
31
        }
32
    }


Create a Pipeline Job in Jenkins and execute the job. It will create the ECS Fargate stack in AWS.

However, this customized Dockerfile is not only for creating ECS Fargate stack, but even other commands can also be triggered which are working fine with the AWSCLI. 

For example:

Shell
 




xxxxxxxxxx
1


 
1
docker run -e \”AWS_ACCESS_KEY_ID=${ACCESS_ID}\” -e \”AWS_SECRET_ACCESS_KEY=${ACCESS_KEY}\” — rm awscli aws ec2 describe-instances — output table


It will show all the EC2 instances in the AWS account and the output will be displayed in tabular format.

Region name for AWS also can be passed using an argument like  -e “region-name”.

Find the code repository in Github here CF_ECS_FARGATE_Jenkins, and check out my DZone profile here.

Keep learning!!!

“Learning never exhausts the mind.” ― Leonardo da Vinci

Further Reading

Using CloudFormation to Set Up Scalable Apps

A Model-Based Approach to CD: From ''Everything as Code" to "Everything is a Model"

Topics:
aws ,aws ecs ,cloud ,cloudformation ,devops ,dockerfile ,everything as code ,jenkins

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}