Microservices: Zero Downtime Deployment and Hot Reconfiguration

2017: Time for a new resolution, and the most important resolution for this year should be to adopt microservices to spend less effort on development and improve your time to market. Nowadays, there are plenty of tools and frameworks at the disposal of the discerning developer to rapidly build microservices. A few examples include Spring Boot, Vert.x, etc.

Once you build your microservices, the next step is to ensure that these frequent deployments do not impact the availability of the microservice. And this is where Zero Downtime Deployment (ZDD) comes in, which allows you to deploy a new version of your microservice without interrupting the operation of the service.

How to Ensure a "Smooth" Deployment

I started working on a Camel microservice, based on Spring Boot, that exposes a REST service. One of the goals is to provide ZDD and a hot reconfiguration feature for the users.

To achieve this goal, I simply have to deploy my microservice on OpenShift, which is based on Kubernetes. What is exciting is that, while successfully handling HTTP requests every second with uninterrupted availability, we have OpenShift perform a zero-downtime rolling upgrade of the service to a new version of the microservice while we're still serving requests per second.

To demonstrate this, we will start from my microservice example here: camel-springboot-rest-ose.

It is assumed a running OpenShift platform is already running. If not, you can find details how to get started (personally, the one that I use is minishift).

Assuming your current shell is connected to OpenShift, you can type the following command to package your microservice, deploy it on OpenShift, and get the output the logs from the running pods:

mvn clean install fabric8:deploy -Dfabric8.deploy.createExternalUrls=true fabric:log

Fabric8:log tails the log of our microservice that's deployed via fabric8:deploy, and we can check when our microservice is started.

By enabling the following property, fabric8.deploy.createExternalUrls, it will create an external URL to reach our REST Endpoint.

To get this URL, you have simply to run the following command:

$ oc get route
NAME                       HOST/PORT                                                   PATH      SERVICES                   PORT      TERMINATION
camel-springboot-rest-os   camel-springboot-rest-os-springboot.192.168.42.111.xip.io             camel-springboot-rest-os   <all>     

Now, we can invoke the REST endpoint of our Camel microservice every second based on the following watch command:

watch -n1 -c ab -n 10 http://camel-springboot-rest-os-springboot.192.168.42.111.xip.io/api/persons/abouchama

Every 1.0s: curl http://camel-springboot-rest-os-springboot.192.168.42.111.xip.io/api/per...  Tue Jan 24 10:38:05 2017

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0 100    85    0    85    0     0    665
      0 --:--:-- --:--:-- --:--:--   669
Hey abouchama --> Enjoy your camel microservices | POD : camel-springboot-rest-os-10-4cxix

In the reply of the REST service, we mention the pod that handles the request inside OpenShift.

To hot configure our Camel route, we will use environment variables. All the endpoints of the Camel route are defined as environment variables properties.

All these environment variables are declared in fabric8/deployment.yml:

spec:
  replicas: 1
  template:
    spec:
      containers:
        -
          resources:
            requests:
              cpu: "0.2"
              memory: 256Mi
            limits:
              cpu: "1.0"
              memory: 256Mi
          env:
          - name: CAMEL_FROM
            value: "direct:getPersonId"
          - name: CAMEL_LOG_MSG
            value: "This request is handled by this POD: {{env:HOSTNAME}}"
          - name: CAMEL_SETBODY
            value: "Hey ${headers.personId} --> Enjoy your camel microservices | POD : {{env:HOSTNAME}} \n"

Now, to update your Camel route, you have only to update the environment variable in the DeploymentConfig:

$ oc env dc/camel-springboot-rest-os CAMEL_LOG_MSG="LOG UPDATED - This request is handled by this POD: {{env:HOSTNAME}}"
deploymentconfig "camel-springboot-rest-os" updated

By running this command, a new deployment will be triggered:

oc get pods -w | grep Running
[aboucham@aboucham camel-springboot-rest-ose]$ oc get pods -w | grep Running
camel-springboot-rest-os-10-4cxix            1/1       Running     0          7m
camel-springboot-rest-os-11-deploy           1/1       Running     0          3s
camel-springboot-rest-os-11-y00ny            0/1       Running     0          3s
camel-springboot-rest-os-11-y00ny            1/1       Running     0          31s

From the OpenShift web console, we will see the following:

Once the new pod is deployed and running, you will see the output of our REST service updated in the watch command with the new pod running without any interruption or impact for our service:

Hey abouchama --> Enjoy your camel microservices | POD : camel-springboot-rest-os-11-y00ny

Scaling

In addition to ZDD, one of the promises of microservices is to allow us to scale easily in order to get better performance from our service. You've got two ways to do that.

Using deployment configuration:

$ oc scale dc/camel-springboot-rest-os --replicas=10
deploymentconfig "camel-springboot-rest-os" scaled

Or using fabric8 start:

mvn fabric8:start -Dfabric8.replicas=10

Conclusion

It all depends on the frequency of your deployments and your production infrastructure. But if you already use multiple microservices, you have everything to gain from Zero Downtime Deployment, even if you only deploy a new version once a month!