Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

How to Set Up Hazelcast IMDG on AWS ECS

DZone's Guide to

How to Set Up Hazelcast IMDG on AWS ECS

Optimize your relational database with this tutorial for installing and configuring open source Hazelcast IMDG.

· Cloud Zone ·
Free Resource

Discover a centralized approach to monitor your virtual infrastructure, on-premise IT environment, and cloud infrastructure – all on a single platform.

Amazon ECS (Elastic Container Service) is a container orchestration service that supports Docker containers. It allows you to create scalable clusters backed up by EC2 Instances. Hazelcast IMDG is distributed in the form of a Docker Image and provides the plugin for auto-discovery in the AWS environment; therefore, it can be simply configured to work on AWS ECS. This post presents this process step-by-step and the source code for this example can be found here.

Step 1: Create AWS ECS Cluster

First, you need a working AWS ECS Cluster. To create it, open AWS ECS Console here. Select “Clusters” and “Create Cluster.” Choose “EC2 Linux + Networking” and click “Next Step.”

Note the following fields as you’ll need them later for the Hazelcast configuration:

  • Cluster name
  • VPC CIDR block

You may also want to set “Security group inbound rules” to allow access to your application from outside AWS.

Step 2: Configure Hazelcast to Work on AWS ECS

You can configure Hazelcast to work on AWS using the hazelcast-aws plugin.

Add the following Maven dependencies:

<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast</artifactId>
    <version>3.11</version>
</dependency>
<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast-aws</artifactId>
    <version>2.3</version>
</dependency>

Then, configure the AWS Discovery Strategy properties. You can do it in two different manners: Java-based configuration or XML configuration. In the code sample, we used the first approach:

public Config hazelcastConfig() {
    Config config = new Config();
    config.getNetworkConfig().getInterfaces().setEnabled(true).addInterface("10.0.*.*");
    JoinConfig joinConfig = config.getNetworkConfig().getJoin();
    joinConfig.getMulticastConfig().setEnabled(false);
    joinConfig.getAwsConfig()
              .setEnabled(true)
              .setProperty("region", "eu-central-1")
              .setProperty("tag-key", "aws:cloudformation:stack-name")
              .setProperty("tag-value", "EC2ContainerService-test-cluster");
    return config;
}


The equivalent XML configuration would look as follows:

<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xmlns="http://www.hazelcast.com/schema/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.11.xsd">
  <network>
    <interfaces enabled="true">
      <interface>10.0.*.*</interface>
    </interfaces>
    <join>
      <multicast enabled="false"/>
      <aws enabled="true">
        <region>eu-central-1</region>
        <tag-key>aws:cloudformation:stack-name</tag-key>
        <tag-value>EC2ContainerService-test-cluster</tag-value>
      </aws>
    </join>
  </network>
</hazelcast>


Note the following parameters:

  • interface: must be the same as “VPC CIDR block”
  • region: must the same as the region in which your cluster is running
  • tag-key, tag-value: ECS automatically tags EC2 instances, so you can use the tags (change “test-cluster” to the name of your cluster); if you don’t specify “tag-key” and “tag-value,” then all your EC2 Instances will be used to form the Hazelcast cluster.

Step 3: Build an Application and Docker Image

mvn clean compile jib:build -Dimage=leszko/aws-ecs-sample


Please change “leszko” to your Docker Hub login. Then, make sure that your image in Docker Hub is public (you can do it on the Docker Hub website).

Step 4: Create AWS ECS Task Definition

Open the AWS ECS Console again and click on “Task Definitions” and “Create new Task Definition.” Select “EC2” and click “Next Step.”

Fill the required fields. Don’t forget to set “Network Mode” to “Host” (that is the only network mode currently supported by the Hazelcast AWS Plugin).

Click “Add container” and fill the required fields.

Add logging to the container specification.

Click “Add” and “Create” and your task definition is created.

Step 5: Start AWS ECS Service

Open your cluster and in the tab “Services,” click “Create.”

Select your task definition and the number of tasks. Click “Next step” a few times and “Create.”

Your service with tasks should be running.

Step 6: Verify that the Application works correctly

Click on any of the running tasks and scroll down to the “Log Configuration” section.

Click “View logs in CloudWatch.” You should see that the Hazelcast Members formed one cluster.

Conclusion

Hazelcast supports automatic members discovery on AWS ECS; however, currently, you’re limited to use the “host” network. This implies that you also have to explicitly specify the network interface pattern. Note that you can use the AWS Discovery plugin for the member-to-member discovery, as well as client-to-member discovery. This allows you to use Hazelcast in both architectures: embedded mode and client-server mode.

Learn how to auto-discover your containers and monitor their performance, capture Docker host and container metrics to allocate host resources, and provision containers.

Topics:
hazelcast ,aws ,aws ecs ,scaling ,installation and configuration

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}