Automated Canary Releasing With Vamp and Jenkins on DC/OS: Part 2

DZone 's Guide to

Automated Canary Releasing With Vamp and Jenkins on DC/OS: Part 2

In Part 2 of this series, we create an initial canary release using DC/OS and Vamp, triggered from a Jenkins CI/CD pipeline.

· DevOps Zone ·
Free Resource

In the first part of this series, we set up our Vamp and Jenkins environments on DC/OS and did our first deploy. If you want to follow along and try out the code examples, you can, however, also read this part to get a general impression of how canary releasing can work for you.

In this part of our series, we will create our initial actual canary release using the combined the power of DC/OS and Vamp, all triggered from a Jenkins CI/CD pipeline. Let's jump in!

Image title

Determining a Versioning Strategy

We need to adopt a versioning strategy to determine what we will deploy and when. For now, we will go for a somewhat simple strategy that mimics "real life:"

  1. Everything in the Master branch is ready for deployment.
  2. New versions are developed in a branch with the version as "1.1.0" and "2.0.0." This allows us to completely prep a release and then merge it to the master.
  3. The version in the package.json file is leading for tagging Docker containers and Vamp artefacts, not the git commit hash or a git tag, although these are also viable options.
  4. Bumping the version after code changes is the responsibility of the developer.
  5. Builds are only for demo purposes triggered manually. In a more advanced scenario, a good strategy would to tag the master branch with the version number and have Jenkins only build and deploy with each new tag.
  6. When deploying to development and testing environments, you could also use git commithashes, where each code push is wrapped in a Docker container and deployed. A downside of this is the possibly very large amount containers in your Docker repo and the need for a strict "cleanup policy" to retire and undeploy no longer relevant containers.

About Merging and Gateways

Our current pipeline script pushes every new version of our app to the public Docker Hub, the Vamp blueprints endpoint, and the Vamp deployment named "simpleservice." When pushing a new version to an existing deployment, something interesting happens.

  • Vamp instructs DC/OS to schedule the new Docker container on the cluster, according to the specs given in the blueprint.
  • Internally, Vamp merges the deployments and a new route in the deployments internal gateway is created. The external gateway remains unchanged.

So if we bump master branch to version 1.1.0 and run our pipeline, a Docker container is created, the blueprint is pushed and the deployment is merged. If you GET the YAML version of Vamp's API endpoint, you would get something very similar to this:

  • One gateway, with naming scheme deployment/cluster/gateway.
  • Two routes, with naming scheme deployment/cluster/service/port with weights of 100% and 0% respectively.
name: simpleservice/simpleservice/web
    weight: 100%
    weight: 0%


The only thing we are left with is to manipulate the weight of each route to gradually release version 1.1.0 of our web app.

An Initial Canary Release With 50/50% Traffic

For our first deploy will hard code the desired outcome of our routing scheme: a 50/50 split between our old and new version. This is reflected in the new  gateway.yml   file in our 1.1.0 branch. We only need to add one call to Vamp to update the gateway api endpoint according to this spec as you can see in the snippet below.

stage('Deploy to Vamp') {
    sh "curl -X POST --data-binary @vamp_blueprint.yml ${vampHost}/api/v1/blueprints -H 'Content-Type: application/x-yaml'"
    sh "curl -X PUT --data-binary @vamp_blueprint.yml ${vampHost}/api/v1/deployments/simpleservice -H 'Content-Type: application/x-yaml'"
    sh "curl -X PUT --data-binary @gateway.yml ${vampHost}/api/v1/gateways/simpleservice/simpleservice/web -H 'Content-Type: application/x-yaml'"


Either copy the snippet above and replace the stage in your existing pipeline script, or just copy and paste the full version from here. Then perform the following steps:

Merge the already prepared 1.1.0 branch to master and push it.

git checkout master
git merge 1.1.0
git push  

After the push finishes, hit the “Build Now” button in Jenkins and you should see the following in your Vamp UI:

Right after that last PUT of the gateway file, your traffic will be split between each version of our service on each request. You can use the pencil icon to pull up the sliders and migrate version 1.1.0 to 100% at your own leisure.

A Second Canary Release With an IP/Browser Filter

The initial 50/50 release instantly exposes all users to our new version. What if we want to have stricter control over who sees what and when? This is exactly what Vamp's conditions are for.

Conditions are basically smart routing filters that allow you to easily dice up traffic using all kinds of request parameters. Conditions are also actual Vamp artefacts that can be stored, referenced, and deleted just like blueprints. This is very handy in a CI/CD context as we can separate what is deployed from how it is deployed. This makes deploys more repeatable and predictable.

Let's create a simple condition named only_firefox by going to the Vamp UI and adding the following YAML snippet:

name: only_firefox
condition: User-Agent = Firefox

Image title

Now, in our code base, we update the  gateway.yml  and add a reference to the condition as shown below. Don’t forget to commit and push this change.

name: simpleservice/simpleservice/web
   weight: 100%
      ref: only_firefox


After re-running our pipeline in Jenkins, you will see that all our traffic goes to 1.1.0, except when you are using a Firefox browser (or strictly speaking, when your request's User-Agent header contains the word "Firefox").

Hopefully, you can see how to start building a comprehensive set of reusable conditions that we can put to use on each release we do.

Towards Sustainable Canary Releases

Every deploy needs a tailor-made gateway.yml that describes an exact starting state. This means that with every new version, you must update this file to reflect the old situation and the desired new situation.

After the initial state, manual intervention is needed to fully migrate to the new version.

Redundant containers are kept running, eating into the resources available in DC/OS. Removing them is a manual task.

We've already gotten quite a bit of automation potential from using Vamp with Jenkins and DC/OS, but as you might have noticed, this setup is not ideal; there are some things that will bite you in the proverbial once you start exercising this pipeline on a regular basis:

Ergo, some things are still manual instead of automated! So how do we leverage the automation potential of Vamp, Jenkins and DC/OS so we can have hands-off, zero downtime canary releases?

In the third part of our series where we will dive into Vamp's CLI and create a truly sustainable canary release pipeline on Jenkins and DC/OS.

automation ,canary release ,continuous delivery ,continuous integration ,dcos ,devops ,docker ,jenkins

Published at DZone with permission of Tim Nolet . See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}