Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Hazelcast for Go Getters, Part 1

DZone's Guide to

Hazelcast for Go Getters, Part 1

In Part 1 of this two-part series, we look at how to setup Goland and the Hazelcast IMDG on your machine/development environment, and their architectures.

· Web Dev Zone ·
Free Resource

Deploy code to production now. Release to users when ready. Learn how to separate code deployment from user-facing feature releases with LaunchDarkly.

Know "Go" but don't know about "Hazelcast"?

Know "Hazelcast" but don't know about "Go"?

This tutorial gets you going with both together, in a follow-along style, assuming you only know one of the two.

Hello World

Naturally, this is a simple example, "Hello World," the bare minimum.

We'll start by installing the tools, then review the architecture, then finally do a bit of coding.

Set Up

This section explains how to set things up on your machine; skip over any bits you're familiar with.

Four short steps here, nice and easy.

"Go" Itself

Firstly, download and install "Go" from https://golang.org/

Once installed, open a new command window and try this command go version

Hopefully, you will see something like this:

$ go version go version go1.9.2 darwin/amd64 $ 

This indicates the version of Go, here it is 1.9.2, and the hardware being used. 'Darwin' means this machine is a Mac.

"Go" structure

The next thing is to create a directory or folder called go, all in lowercase, in your home directory. This is the convention, best not to deviate from it unless you really know what you're doing.

Then, create an environment variable named GOPATH, all in uppercase, that points to this directory and store it in your profile so it is available each time you log in.

"Hazelcast Go" Client

Finally, we want to download Hazelcast's client for Go. Hazelcast here will be a client-server architecture, we'll cover this later on. For now, the server side is Java, so the client side we want to be Go.

The command here is go get, hence the title of this tutorial. We ask Go to get the code from GitHub.

From the go directory, run this command:

go get github.com/hazelcast/hazelcast-go-client 

Or this with the -v flag for verbose output:

go get -v github.com/hazelcast/hazelcast-go-client 

This will pull down the Hazelcast client for Go from GitHub, from here, and set up the necessary Go directory structure.

Verification

Go will place the Hazelcast client for Go in a subdirectory following a specific structure.

You should see at least these folders

├── go/
├── go/pkg/
├── go/src/
├── go/src/github.com
├── go/src/github.com/hazelcast
├── go/src/github.com/hazelcast/hazelcast-go-client

The pkg folder holds output packages.

The src folder holds the source code, in a subdirectory for GitHub code, then a subdirectory for the Hazelcast GitHub user, and then a subdirectory for the Hazelcast client for the Go repository.

If you see this, we are done with Go set up.

Hazelcast Set Up

Three simple steps.

Hazelcast is a Java application, so you need a Java implementation.

The best place to get this from is OpenJDK, an open source implementation of Java. Or commercially, from Oracle.

Either way, take the latest version of Java 9 available for your platform, and this should come with an installer.

The Java language has two variants alive currently, Java 8 and Java 9, but Java 8 will be retired first so Java 9 is a better choice.

If offered a JRE (Java Runtime Environment) or JDK (Java Development Kit), either will do here. The development kit adds compilers to the runtime, but we don't use them in this example.

Hazelcast

Next download Hazelcast.

You can get this from a number of places, in this case, try here.

You want "Hazelcast IMDG." IMDG here is short for In-Memory Data Grid.

The most recent release is shown at the top. Select the ZIP or TAR, download and unpack. You can put it anywhere, for now, so your downloads folder is as good a place as any.

You should see something like this, assuming a version of 3.9:

├── hazelcast-3.9/
├── hazelcast-3.9/bin
├── hazelcast-3.9/code-samples
├── hazelcast-3.9/demo
├── hazelcast-3.9/docs
├── hazelcast-3.9/lib
├── hazelcast-3.9/license
├── hazelcast-3.9/mancenter

Verification

Firstly, run the java -version command to see the Java version.

You will likely need to open a new window to ensure Java is known to the command interpreter, but should then get version output. The exact wording will likely differ.

$ java -version
java version "9.0.1"
Java(TM) SE Runtime Environment (build 9.0.1+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.1+11, mixed mode)
$ 

If Java is found, then you can try Hazelcast, using the start.sh script in Hazelcast's bin folder.

This will give a lot of output, as Hazelcast is quite verbose by default about what it is doing, but, towards the end, you should see messages like:

INFO: [10.37.217.195]:5701 [dev] [3.9] 

Members {size:1, ver:1} [
    Member [10.37.217.195]:5701 - 57a58f7f-4453-4a6f-8e89-bf1a464e46b1 this
]

INFO: [10.37.217.195]:5701 [dev] [3.9] [10.37.217.195]:5701 is STARTED

This message indicates you have a cluster (of one!) Hazelcast servers running on your machine. In this case, the Hazelcast server is on the machine with IP address 10.37.217.195 and port 5701.

Architecture

Refer to the following diagram and subsequent explanation

The Servers - Hazelcast running Java

Hazelcast's software here is the In-Memory Data Grid or IMDG.

A Hazelcast IMDG server is a Java process and therefore runs in a Java Virtual Machine (JVM). In this diagram, these are represented by the orange box (the JVM) hosting the IMDG server.

Four of these are shown. So there are four Hazelcast servers acting as a cluster here, doing 1/4 of the work each. This is easily scalable, add a fifth server and the processing and storage capacity will automatically increase by 25%.

In this diagram, Hazelcast has a data storage area named "card" to hold playing cards. This is a superset of a Java Map and similar to a database table.

One major difference is the storage is spread across the available servers - seven light green boxes indicate the "card" storage area has been split into seven parts. This is how scaling is achieved when more servers join the cluster, the available parts are rearranged as evenly as possible. Seven parts spread across four servers is just for the diagram, the default if two hundred and seventy-one parts, but that would be too cluttered to visualize.

A second major difference is the data is mirrored. In the diagram each light green box for part of the "card" storage area has a counterpart dark green box somewhere else. Should a process fail and a light green box be lost, the dark green pair can be brought into service and no data is actually lost. Naturally, this is configurable, you can have as many backup copies as you feel appropriate.

These JVMs can be anywhere so long as the network between them is fast enough. They could all be running on the same physical machine. This is easy for development but not too clever for production, as should the physical machine fail it takes every JVM offline.

So, this is the IMDG in a nutshell. A group of Java processes joins together to form a cluster taking an equal share of data storage and processing. Data is held in memory, so it's very fast to access. The storage containers are Java JVMs, so data processing is handled by familiar Java classes that you can write. Scalability and resilience are enabled by default.

The Client - Go

Instead, the Hazelcast cluster can be regarded as a server layer, and a Go routine can connect to it as a client to retrieve data, process retrieved data, and to save retrieved data back to the cluster.

This is represented in the diagram by the Go routine in the top left. This is one Go routine connected to the cluster. Several Go routines can connect to cluster at once, all reading and writing data safely. Indeed, if you want multiple clients connected to the Hazelcast cluster can be in a mix of languages - Node.js, Java, Go and so on.

As far as Go is concerned, the presence of Hazelcast is abstracted. Functions are provided such as "GetMap" to return a reference to a map, and "Get" to get an item from that map. The Go routine does not need to concern itself that the map is stored elsewhere - across multiple Java JVMs - as this is handled by Hazelcast.

In the diagram, arrows show that the Go routine opens a connection to every Hazelcast server at once. If data is needed from part 1 of the map distributed across the cluster, this will use the appropriate connection, in this case, to the second server from the left. If data is needed from part 2, the connection to the second server from the right is used. So each retrieval only needs one network interaction, regardless of how many servers are used to spread the data - network transfer is reduced to the minimum. In this second case, asking for data from part 2, the required data record may not be present, just like any storage system a request may return not found.

This is the standard read-mutate-save cycle. The Go client retrieves data directly from the server that has the particular item, the Go client does something with the data to change it perhaps, and then the Go client sends the data back to the Hazelcast server to update the master copy.

If the data is big or the network is poor, it can be more efficient to send the processing function to the data to modify it in situ.

For now, the closing point on this section is that the Go client is acting here as a proxy. Data is copied across the network from Hazelcast's memory to the Go routine's memory. If something else changes the copy held in Hazelcast, the Go routine won't (by default!) know this has happened. If the Go routine does not save the data back to Hazelcast after changing it, the master copy held on Hazelcast won't change and no other client with have visibility of the update.

That's all for today. Tune back in tomorrow when we'll cover coding and clustering!

Deploy code to production now. Release to users when ready. Learn how to separate code deployment from user-facing feature releases with LaunchDarkly.

Topics:
hazelcast ,go ,web dev ,imdg ,golang app development

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}