Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Continuous Delivery With Kubernetes, Docker, and CircleCI

DZone's Guide to

Continuous Delivery With Kubernetes, Docker, and CircleCI

I decided to create a small side-project to keep my Kubernetes skills fresh — oh, and to use absolutely every technology I've ever used!

· DevOps Zone
Free Resource

Download the blueprint that can take a company of any maturity level all the way up to enterprise-scale continuous delivery using a combination of Automic Release Automation, Automic’s 20+ years of business automation experience, and the proven tools and practices the company is already leveraging.

In this article, I'll walk you through my experience setting up a Continuous Delivery pipeline using (mostly) open and free tools.

I've spent a lot less time writing code over the last year and decided to create a small side-project to keep my Kubernetes skills fresh — oh, and to use absolutely every technology I've ever used (Deployment, Kubernetes, Nginx, Docker, CircleCI, Mongo, Postgres, Java 8, Scala 2.11, Node, HTML, CSS, JavaScript, Ruby, Spring Boot, Playframework 2, jQuery, Phaser [PIXIJS], JUnit, ScalaTest, WDIO, Selenium WebDriver, RestAssured, Gatling, Git/GitHub, Middleman, Jekyll, Swagger 2, urm Lisp? Cobol? Prolog? TCL?).

The project in a system for writing games, and has six main components:

Architecture

Each component has a role to play:

  • NGINX does TLS termination and serves the web content (HTML/CSS/JS).
  • The router does authentication, message enrichment/routing.
  • The wallet backs onto a Postgres database and allows money to be added and removed from wallets.
  • A game that backs onto a Mongo database, and models games.

I wanted to deliver this onto a server automatically, so I created this pipeline:

CI

This means that every commit is built, tested, packaged, deployed, and smoke and load tested.

I said this is nearly free. The one thing I paid for was some cloud-hosted servers. I used Digital Ocean, as they have a fantastically easy-to-use interface and a variety of data centers. Kubernetes needed 3x 2CPU/2GB machines, which set me back $60 a month. You can power them down when you're not using them (using the Digital Ocean API and a curl script).

Getting Kubernetes set up from scratch (the "hard way") takes a long time and requires a lot of technical expertise and patience. Instead, I used Stackpoint to do this for me. To do this, I had to create an API token for Digital Ocean, sign up for Stack Point, and tell it what the token is. It spun up a Kubernetes cluster in about 15 minutes.

Once it's started up, you're emailed an kubeconfig file to use with kubectl.

I build my applications and deploy them to Docker Hub. I don't use any specialist tools to do this, just Maven, SBT, and the core Docker tools. You need to have DOCKER_USER, DOCKER_EMAIL, DOCKER_PASS — your Docker Hub username, email, and password. Here's a build script for a standalone JAR Maven project:

mvn package -DskipTests
docker build --rm=false -t alexec/wallet:$CIRCLE_BUILD_NUM -t alexec/wallet:latest .
docker login -e $DOCKER_EMAIL -u $DOCKER_USER -p $DOCKER_PASS
docker push alexec/wallet

For speed, I used the Alpine base images. The Dockerfile for the Java apps is:

FROM openjdk:8-jre-alpine
ADD target/wallet.jar /
ENV JAVA_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -jar /wallet.jar" ]

The process for building standalone Playframework is a bit more complex, as Playframework does not produce a standalone JAR normally:

sbt dist
unzip -d target/universal target/universal/router-1.0.0-SNAPSHOT.zip
docker build --rm=false -t alexec/router:$CIRCLE_BUILD_NUM -t alexec/router:latest .
docker login -e $DOCKER_EMAIL -u $DOCKER_USER -p $DOCKER_PASS
docker push alexec/router
FROM openjdk:8-jre-alpine
ADD target/universal/router-1.0.0-SNAPSHOT/ /
RUN rm -f RUNNING_PID
ENV JAVA_OPTS=""
ENV APPLICATION_SECRET=changeme
ENTRYPOINT [ "sh", "-c", "java -cp 'conf:lib/*' play.core.server.ProdServerStart"]

These build each commit on CircleCI. I particularly like CircleCI's clean user interface and its "insights" features. I choose it in preference to the popular TravisCI.

I need to chain my builds into a pipeline, but CircleCI doesn't provide that out of the box. It has something better (IMHO), an API you can call to start a build. Each build ends up invoking this command:

curl -fv -u $CIRCLE_TOKEN: -X POST -d '{\"build_parameters\": {\"COMPONENT\": \"router\"}}' -H 'Content-Type: application/json' https://circleci.com/api/v1.1/project/github/phoebus-games/deploy/tree/master

CIRCLE_TOKEN is a CircleCI API token (you can create this using Personal API Tokens). This starts the deploy build add passes the name of the component to build. This build does some clever stuff, which I'm going to include in ill is gory glory:

dependencies:
  override:
  - curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
  - chmod +x kubectl
  - sudo apt-get -y install gnupg
compile:
  override:
  - echo $GPG_KEY | gpg -d --passphrase-fd 0  --batch --yes kubeconfig.gpg > kubeconfig
test:
  override:
    - "true"
deployment:
  prod:
    branch: master
    commands:
      - env KUBECONFIG=kubeconfig PATH=$PATH:. kubectl replace --force -f $COMPONENT.yml
      - env KUBECONFIG=kubeconfig PATH=$PATH:. kubectl get pods -o name|grep "$COMPONENT\|web"|env KUBECONFIG=kubeconfig PATH=$PATH:. xargs kubectl delete
      - "curl -f -u $CIRCLE_TOKEN: -X POST https://circleci.com/api/v1.1/project/github/phoebus-games/smoke/tree/master"

Phew! OK, so what does this do?

  • Installs kubectl so that I can deploy Kubernetes workloads, and gnupg to decrypt the kubeconfig file.
  • Decrypts the kubeconfig file by passing the password in via stdin.
  • Skips tests!
  • Deploys the component.
  • Re-deploys Nginx (this is so that it can get the updated IP of the service).
  • Kicks of the smoke test.

GPG_KEY is a random string. I used this to encrypt kubeconfig, as there are no "secrets" in CircleCI.

Finally, a successful smoke test kicks off a load test.

Tips

  • Integration test with a real database running locally. This is super easy with Homebrew on the Mac.
  • Separate the building of your component from the building of the Docker image. Use different tools.
  • Separate deployment from build and then take advantage of the Docker packaging to reuse the same deployment script.
  • Don't bother trying to run it all locally. Use Wiremock to simulate your dependencies when doing integration testing.

Download the ‘Practical Blueprint to Continuous Delivery’ to learn how Automic Release Automation can help you begin or continue your company’s digital transformation.

Topics:
kubernetes ,docker ,circleci ,devops ,continuous delivery ,tutorial

Published at DZone with permission of Alex Collins. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}