Provisioning AWS CodePipeline with CloudFormation

since our entire focus at stelligent is to help our customers apply continuous delivery in amazon web services (aws), we were really excited when we learned that aws cloudformation added support for aws codepipeline . i spent some time upgrading our dromedary demo scripts to incorporate this new functionality. this article describes the process we applied in making the necessary changes along with some bonus functionality we were able to implement a more fully automated solution – thanks to corey at aws support .

when the aws codepipeline team released its continuous delivery service in july 2015, they provided a custom json-based dsl for it which we used to automate the provisioning of codepipeline itself but we didn’t get it into the 100% automated state for which we’d been hoping. it was close, but were “chomping at the bit” for the time when they’d begin providing cloudformation support. thankfully, now’s the time.

a couple of stelligent’s engineers wrote the first version of the demo that we used at the “infrastructure as code” breakout session as part of aws re:invent 2015 so they’d already been through some of the things i learned while adding and updating our scripts.

below, you’ll find the steps for getting this demo running on your own aws account. all of the code is freely available and open source.

running from cloudformation

download the raw file from https://raw.githubusercontent.com/stelligent/dromedary/master/pipeline/cfn/testdrive.json and upload it into cloudformation to run the stack. enter the required parameters. in particular: ec2sshkeyname , githubtoken, githubuser and prodhostedzone . see the figure below and complete the rest of the steps to launch the stack. it’ll take approximately 20 minutes to launch the stack and then launch a codepipeline pipeline instance that launched an application stack. you can see the readme for other ways to run this template from cloudformation.

once all the required aws resources are bootstrapped, it automatically launches a pipeline instance in codepipeline as shown below.

running from the command line

you’ll need to run the command below from a computer on which git, dromedary and the aws cli has been installed.

step 1: clone the dromedary repo
git clone -b master https://github.com/stelligent/dromedary.git

step 2: change the directory
cd ~/dromedary

step 3: run the bootstrap command
you’ll need to supply two commands to the bootstrap-all.sh bash script. the first argument is the subdomain and domain for the hosted zone that you’ve already configured and have access to in aws route 53 . the second argument is your github token. you’ll need to configure this token by going to https://github.com/settings/tokens . since this provides access to your github account, you’ll need to treat it as secret information. the cloudformation template applies a noecho property to the parameter so that the value is not displayed in the output of the cloudformation stack.

./bin/bootstrap-all.sh dromedary.yourdomain.comyourgithubtoken

for example:
./bin/bootstrap-all.sh dromedary.yourdomain.com 5b298a1123546544f31f7b4ebd99a1531a6578af .

cloudformation template for codepipeline details

the steps i went through to create this cloudformation template were fairly straightforward. first, i started with a cloudformation template that i had implemented for another effort and removed most everything except for the core structure including the awstemplateformatversion , an empty parameters block and an empty resources block. the core resources type for codepipeline is – as you might’ve guessed - aws::codepipeline::pipeline . after this, i got the name of an existing pipeline that we’d created using the aws codepipeline json dsl from our aws codepipeline and ran this command:

aws codepipeline get-pipeline --name your_pipeline_name > pipeline.json

this provided a json structure with all the stages and actions already defined in code so i was able to copy and paste within my aws::codepipeline::pipeline resources construct in the cloudformation template. since the codepipeline dsl produces a slightly different case than cloudformation, i needed to update the letter case for several properties to conform to the cloudformation standard.

then, i added several parameters to the template including githubtoken and githubuser . finally, i ran through several testing scenarios and updated the bootstrap shell script and the cloudformation template that calls the bootstrap script. be sure to run through all the steps in readme as well since it’s important that you enable security for jenkins.

summary

essentially, until aws implements a service in cloudformation, we don’t really consider it a service we can use in the manner we like so we’re really happy that codepipeline can now be provisioned in cloudformation. moreover, when implementing the codepipeline provisioning in cloudformation, we configured it so that all of codepipeline including the github configuration is automated so there are no manual steps anymore (except for enabling jenkins global security ). if you have any questions or comments, you can contact us at info@stelligent.com or consult the contact information at https://github.com/stelligent/dromedary.