How to Use Embedded Hazelcast on Kubernetes

Hazelcast IMDG is a perfect fit for your (micro)services running on Kubernetes since it can be used in the embedded mode and therefore scale in and out together with your service replicas. This blog post presents a step-by-step description of how to embed Hazelcast into a Spring Boot application and deploy it in the Kubernetes cluster. The source code for this example can be found here.

Step 1: Configure Hazelcast to Work on Kubernetes

Make sure you have the following Maven dependencies:

<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast-all</artifactId>
    <version>4.1</version>
</dependency>

Then, configure the Kubernetes Discovery Strategy. You can do it in 3 different manners: YAML configuration, XML configuration, or Java-based configuration. Let's use the first approach and create the file src/resources/hazelcast.yaml: 

hazelcast:
  network:
    join:
      kubernetes:
        enabled: true

The equivalent XML configuration would look as follows:

<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xmlns="http://www.hazelcast.com/schema/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-4.1.xsd">
  <network>
    <join>
      <kubernetes enabled="true"/>
    </join>
  </network>
</hazelcast>

Note that this configuration will form a Hazelcast with all Hazelcast instances assigned to services in the current namespace. If you want to filter the instances, use the properties described here.

Step 2: Build an Application and Docker Image

Now, you need to build the application and publish it as a Docker image. First, let’s build the project.

mvn package

Then, we can create the following Dockerfile.

FROM openjdk:8-jre-alpine
COPY target/*.jar app.jar
ENTRYPOINT ["java","-jar","app.jar"]

Finally, we’re ready to build and push our Docker image.

docker build -t leszko/hazelcast-embedded-sample .
docker push leszko/hazelcast-embedded-sample

Note that you need to change leszko to your Docker Hub account name.

Step 3: Grant Access to Kubernetes API

Hazelcast uses Kubernetes API for auto-discovery. That is why you need to grant certain roles to your service account. You can do it by creating the following “rbac.yaml” file.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: default-cluster
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
- kind: ServiceAccount
  name: default
  namespace: default

Then, apply it into your Kubernetes cluster.

kubectl apply -f rbac.yaml

Step 4: Deploy the Application

Create “deployment.yaml” with Deployment and Service which will use the image you pushed to Docker Hub.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hazelcast-embedded
spec:
  replicas: 2
  selector:
    matchLabels:
      app: hazelcast-embedded
  template:
    metadata:
      labels:
        app: hazelcast-embedded
    spec:
      containers:
        - name: hazelcast-embedded
          image: leszko/hazelcast-embedded-sample
          imagePullPolicy: Always
          ports:
          - containerPort: 5701
          - containerPort: 8080

---

apiVersion: v1
kind: Service
metadata:
  name: hazelcast-embedded
spec:
  type: LoadBalancer
  selector:
    app: hazelcast-embedded
  ports:
  - name: hazelcast
    port: 5701
  - name: app
    port: 8080

Then, to deploy an application, run the following command:

kubectl apply -f deployment.yaml

Step 5: Verify that The Application Works Correctly

You can check that the Deployment and Service were created.

$ kubectl get all
NAME                                      READY     STATUS    RESTARTS   AGE
pod/hazelcast-embedded-57f84c545b-64tnk   1/1       Running   0          2m
pod/hazelcast-embedded-57f84c545b-jjhcs   1/1       Running   0          45s

NAME                         TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                         AGE
service/hazelcast-embedded   LoadBalancer   10.19.251.145   104.154.43.142   5701:32302/TCP,8080:31613/TCP   2m

NAME                                       DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/hazelcast-embedded   2         2         2            2           2m

NAME                                                  DESIRED   CURRENT   READY     AGE
replicaset.extensions/hazelcast-embedded-57f84c545b   2         2         2         2m

NAME                                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/hazelcast-embedded   2         2         2            2           2m

NAME                                            DESIRED   CURRENT   READY     AGE
replicaset.apps/hazelcast-embedded-57f84c545b   2         2         2         2m

In the logs for PODs, you should see that the Hazelcast members formed a cluster.

$ kubectl logs pod/hazelcast-embedded-57f84c545b-jjhcs
 ...
 Members {size:2, ver:4} [
         Member [10.16.2.6]:5701 - 33076b61-e99d-46f2-b5c1-35e0e75f2311
         Member [10.16.2.8]:5701 - 9ba9bb61-6e34-460a-9208-c5a644490107 this
 ]
 ...

Then, you can access the application, by its EXTERNAL-IP.

That’s it! If you would like to learn more about scaling Hazelcast IMDG on Kubernetes, here is another post with step-by-step instructions.