Cloud Native Data Grids: Hazelcast IMDG With Kubernetes

The term "cloud-native" is applied to applications that fit naturally into cloud environments, such as those orchestrated by Kubernetes. Architecturally, this means the application is modularized. It is a service composed of multiple module instances, deployed collectively across multiple execution hosts.

The key point is around the multiple instances. The number can be adjusted to control service capacity, and the failure of any one instance does not necessarily result in a service outage.

In-Memory Data Grids (IMDGs) are a natural fit into this methodology. An IMDG is a collective service formed of multiple processes clustering together, to provide data hosting and data processing operations. For Hazelcast's IMDG (which we will use in the following examples), these processes are Java virtual machines (JVMs). On Kubernetes, these JVMs run in containers within pods, and the Hazelcast IMDG service is formed using multiple pods.

A Hazelcast IMDG is a collection of Java processes that join together to provide a communal data service. Each of these storage processes is a Java virtual machine, running in a Docker container, which is the only container in a Kubernetes pod. Kubernetes runs multiple pods to make the IMDG as a whole.

Kubernetes is responsible for managing the pods and finding machines on which to start the pods to build the Hazelcast IMDG of the requested size. Hazelcast will then spread the data across the available pods so that each one stores some of the data.

Both capacity and throughput are scalable. If the pod count is increased or decreased, the data records are rearranged across the pods to balance the load. If the data capacity needs to double, doubling the number of pods achieves this demand. Each pod can host the same number of data records as before, but now there are twice as many of them.

If the data throughput needs to double, doubling the number of pods achieves this. The same data volume on a grid twice this size halves the amount of data per pod. With half as much data as before, each pod can cope with twice the number of requests per record.

This is a preview of the Cloud Native IMDGs: Hazelcast IMDG With Kubernetes Refcard. To read the entire Refcard, please download the PDF from the link above.