Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Deploying Apache Ignite in Kubernetes on Microsoft Azure

DZone's Guide to

Deploying Apache Ignite in Kubernetes on Microsoft Azure

Apache Ignite's most recent release includes a Kubernetes integration. See it in action as you learn to run a cluster on Microsoft Azure.

· Cloud Zone ·
Free Resource

Discover a centralized approach to monitor your virtual infrastructure, on-premise IT environment, and cloud infrastructure – all on a single platform.

The recent Apache Ignite 1.9 release provided an integration with Kubernetes that automates deployment, scalability, and management of an Apache Ignite cluster running under its supervision. The whole idea of the integration is to enable Apache Ignite nodes auto-discovery in Kubernetes so that they can interconnect with each other to form a distributed cluster. This capability is supported by a special Kubernetes IP finder that gathers IP addresses of all running Apache Ignite nodes and references these IP address at the time a new Apache Ignite node needs to join the cluster.

The Apache Ignite community prepared Kubernetes Deployment Getting Started Guide that demonstrates how to deploy a cluster on a local machine using Minikube. In this blog post, it will be shown how to do the same but for a cloud environment such as Microsoft Azure.

Creating a Kubernetes Cluster on Microsoft Azure

Before we can deploy an Apache Ignite cluster in Kubernetes we need to allocate virtual machines and prepare a Kubernetes cluster on Microsoft Azure. The fastest way to accomplish this step is by referring to the excellent walkthrough prepared by Microsoft. 

Once you have completed a part of the walkthrough that explains how to create the Kubernetes cluster and connect to it from a local machine, run a proxy to the Kubernetes API server using the command below:

kubectl proxy


After that, make sure that Kubernetes UI is available on the local machine by opening the following URL in a favorite browser:

http://localhost:8001/ui


You should see a page like that once you go to Nodes tab:

Kubernetes nodes prepared to host Apache Ignite

Creating Kubernetes Ignite Lookup Service

The Kubernetes IP finder requires us to create a special Kubernetes service to which the IP finder connects to relying on Kubernetes API in order to get the IP addresses of all the alive Apache Ignite nodes (pods in terms of Kubernetes). 

Every time a new Apache Ignite node is started, the IP finder will connect to the service and will retrieve addresses of already running Ignite nodes. Using these addresses, the new node will be able to discover the rest of the cluster nodes and join the Apache Ignite cluster.

First, to create the service we need to prepare ignite-service.yaml file with the content below:

apiVersion: v1
kind: Service
metadata:
  name: ignite
spec:
  clusterIP: None
  ports:
    - port: 9042 # custom value.
  selector:
    app: ignite


Second, instantiate the service using kubectl create -f ignite-service.yaml command and go to Kubernetes UI Services tab to check that the Ignite service is up and running:

Apache Ignite Service and Kubernetes

Deploying Apache Ignite Cluster

To deploy an Apache Ignite cluster in Kubernetes we need to prepare two things: Apache Ignite configuration with enabled Kubernetes IP finder and Kubernetes YAML configuration for Apache Ignite pods (nodes).

As for the first, we will simply reuse an existing Spring XML configuration available on GitHub. The YAML configuration in its turn will pass this Apache Ignite Spring XML configuration to Apache Ignite docker image via CONFIG_URIparameter as it's shown below:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: ignite-cluster
spec:
  # Start two Ignite nodes by default.
  replicas: 2
  template:
    metadata:
      labels:
        app: ignite
    spec:    
      containers:
        # Custom Ignite pod name.
      - name: ignite-node
        image: apacheignite/ignite:1.9.0
        env:
        - name: OPTION_LIBS
          value: ignite-kubernetes
        - name: CONFIG_URI
          value: https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube.xml
        ports:
        - containerPort: 11211 # REST port number.
        - containerPort: 47100 # communication SPI port number.
        - containerPort: 47500 # discovery SPI port number.
        - containerPort: 49112 # JMX port number.


Create the ignite-deployment.yaml file with the content above and deploy an Apache Ignite cluster in Kubernetes executing kubectl create -f ignite-deployment.yaml command. To learn more about the configuration parameters used in ignite-deployment.yaml, refer to this section of Apache Ignite Kubernetes getting started guide.

Make sure that the Apache Ignite pods have been deployed successfully by clicking on Pods tab of Kubernetes UI:

Apache Ignite Pods in Kubernetes

 Check the logs of one of the nodes confirming that the nodes, eventually, have discovered each other in Kubernetes environment on Microsoft Azure:

Apache Ignite Logs

 At the end, let's instruct Kubernetes to scale out the Apache Ignite cluster from two to five nodes executing the following command:

kubectl scale --replicas=5 -f ~/kubernetes_dev/azure/ignite-deployment.yaml


Refresh the Kubernetes UI Pods tab to see that the cluster has been scaled out automatically:

Apache Ignite Scalability

Conclusion

As demonstrated, the new Apache Ignite and Kubernetes integration allows for quick and easy deployment and management of a distributed Apache Ignite cluster under Kubernetes supervision. Microsoft Azure is just an example deployment scenario and you're free to deploy a combined Kubernetes and Apache Ignite cluster on any cloud platform or on-premise.

Learn how to auto-discover your containers and monitor their performance, capture Docker host and container metrics to allocate host resources, and provision containers.

Topics:
apache ignite ,kubernetes ,microsoft azure ,distributed computing ,cloud ,tutorial

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}