Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Hazelcast for Go Getters, Part 2

DZone's Guide to

Hazelcast for Go Getters, Part 2

To wrap up this two-part series, we look at how to connect our Golang-based application to a Hazelcast server and retrieve data.

· Web Dev Zone ·
Free Resource

Learn how to build stream processing applications in Java-includes reference application.

Welcome back! If you missed Part 1, you can check it out here

Coding

If both "Go" and "Hazelcast" are successfully installed, and you have a rough understanding of the architecture, then it's time for Hello World.

Outline

Only one Hazelcast server isn't typical. In the bonus section towards the end, we expand the cluster to two!

Similarly, running clients consecutively isn't typical either. Multiple clients can be connected to the cluster at once, all doing data operations.

The Hazelcast Half

This example has no coding for the Hazelcast half. We use the pre-build Hazelcast start script provided in the download.

In the Hazelcast download there is a folder bin with a script in it called start.sh and stop.sh (and Windows variants, start.bat and stop.bat).

In the earlier Hazelcast verification step you will have run the start script, so Hazelcast will already be running. Proof of this is to look in the hazelcast_instance.pid file in this folder and see if the listed process id is indeed running.

If it is running, use the stop script to stop it. This should close that running Hazelcast process, and remove the hazelcast_instance.pid file.

Now, no Hazelcast server is running, so use the start script to start one. Towards the end, you will see output like

Members {size:1, ver:1} [
    Member [192.168.1.156]:5701 - f4855568-dcd6-4cbc-b231-6f8af5c72487 this
]

This indicates a process has started, on IP address 192.168.1.156 port 5701, and as it is the only one listed the cluster size is one member.

This example does not do server-side processing, so we don't need to pay attention to the Hazelcast server logs for any insights.

The Go Half

There are three programs, imaginatively named one.gotwo.go and three.go.

All three are clients of the Hazelcast server, and, for this example, we run them in sequence, though they could be run concurrently.

ONE.GO

From the command line, in the directory with the one.go program ($GOPATH/src/github.com/hazelcast-go-client/samples/hello-world/one) run this command:

go run one.go

This will request that Go run the one.go program.

This program does very little except try to connect to the Hazelcast server on the host:port pairing of 127.0.0.1:5701, to prove connectivity.

One of two things will happen, depending on whether the Hazelcast server is running. Try this both with and without the Hazelcast server running to see what happens.

Failure

If the Hazelcast server is not running, the Go client will fail to connect to it.

You should see an error message much like this

Cluster name:  dev
Cluster discovery:  [127.0.0.1:5701]
2017/11/21 14:43:45 New State :  STARTING
2017/11/21 14:43:46 the following error occured while trying to connect to cluster:  target is disconnected
2017/11/21 14:43:47 the following error occured while trying to connect to cluster:  target is disconnected
2017/11/21 14:43:47 New State :  SHUTTING_DOWN
2017/11/21 14:43:47 New State :  SHUTDOWN

The client will try to connect a couple of times, then give up and the routine ends.

Success

If the Hazelcast server is running, the Go client will connect to it.

So this time you should see this:

Cluster name:  dev
Cluster discovery:  [127.0.0.1:5701]
2017/11/21 14:53:39 New State :  STARTING
2017/11/21 14:53:39 Registered membership listener with Id  c06ec961-0159-481b-a6ee-fa00c145f3b9
2017/11/21 14:53:39 New State :  CONNECTED
2017/11/21 14:53:39 New State :  STARTED
2017/11/21 14:53:39 New State :  SHUTTING_DOWN
2017/11/21 14:53:39 New State :  SHUTDOWN

Discovery

The Go client is configured in one.go to try to connect to 127.0.0.1:5701.

Obviously, this fails if there isn't a Hazelcast server on that host:port.

More usually, you run multiple Hazelcast servers to form a cluster. In this case, you can still configure the Go client with host:port of one of the Hazelcast servers. It's a better idea to list the host:port pairings for two or three of the Hazelcast servers than just one, in case that one is offline. But you don't need to list them all if you get an answer from any of the listed Hazelcast servers, the answer includes the location of all the other Hazelcast servers in the cluster.

TWO.GO

Run:

go run two.go

from the hello-world/two folder in the Go source hierarchy.

You should get output like:

Map 'greetings' Size before 0
Map 'greetings' Size after 5

What Is This Doing?

What two.go does is connect to the Hazelcast servers, of which there is currently just one.

The data structure being used is a key-value store. Both the key and the value here are strings. In Hazelcast and in Java, in general, this structure is called a map. Other technologies call it a properties list, or a dictionary.

These maps are named, you can have several of them, and use the names to distinguish.

Five data records are inserted.

For the key " English," the string value " hello world " is saved.

This is repeated for French, German, Italian and Spanish, making for five data records.

Optional

Run two.go a second time, the output should look like:

Map 'greetings' Size before 5
Map 'greetings' Size after 5

We are using Hazelcast's map structure, which is a key-value store.

The first run of two.go added five new keys to an empty Hazelcast, so the before count was 0 and the after was 5.

This second run of two.go inserts the same five keys. So there were 5 before, and as the same 5 are written there are still 5. The writes here replace the values that were present.

Specifically, the Go code is doing a "Put" operation on the map which inserts the data if not present and replaces the data is present. This is good enough for most purposes but there are other versions,  PutIfAbsent" and " Replace" that allow a finer degree of control depending on if the data is already present. See IMap for more details

What Is This Doing (Contd.)?

Hazelcast uses a lazy-creation strategy.

When a Go client requests a map with the name "greetings," it will be created if it doesn't already exist.

This is how you can run two.go more than once and keep retrieving access to the same map. At the first reference, the map is created and access to it is returned. At the second reference, it already exists and, again, access to it is returned.

Optimizations

The five data records are sent individually, using "Put."

So there are ten network interactions. Five send and receive pairs.

There is also a bulk send operation, "PutAll" that sends a collection of inserts together in a batch. This would be two network interactions, so more efficient for the same effect.

THREE.GO

Run:

go run three.go

from the hello-world/three folder in the Go source hierarchy.

You should get output like this:

 -> 'Spanish'=='hola mundo'
 -> 'German'=='hallo welt'
 -> 'Italian'=='ciao mondo'
 -> 'English'=='hello world'
 -> 'French'=='bonjour monde'
[5 records]

You should definitely get five records back. We've not bothered to sort, so the sequence you have them displayed will likely vary.

What Is This Doing?

The main operation here is "KeySet" on the map. This returns a set of keys in the map, similar to selecting the primary key column in a relational database. Every key is present in the set exactly once.

Thereafter, the code iterates through the set of keys, retrieving each value to print.

Danger Area

Earlier we mentioned that it would be more efficient for two.go to send all data at once using the "PutAll" bulk operation.

The counterpart for "PutAll" bulk sending is "GetAll" for bulk receiving, and three.go could use this.

"GetAll" is indeed more efficient for bulk receiving than to iterate and get each item individually. However, there is a risk to be aware of.

Optimizations

The output order is effectively random, which is a bit irritating.

However, the keys are strings. The set of keys retrieved could easily be sorted before displaying the value for each one.

Bonus Step - Clustering

As per above, typical usage of Hazelcast is to run multiple servers that join together to form a cluster, providing resilience and scalable storage.

The start.sh script in Hazelcast's bin folder creates one Hazelcast instance and logs its process id in a *.pid file for later use.

Clean Start

Firstly, we want to ensure no Hazelcast servers are running.

The easiest way is just to kill off any Java processes from the OS. On Windows, this might be with taskkill, on Mac with kill -9.

Also, make sure the hazelcast_instance.pid is removed.

Start a First Hazelcast Server

Use the start.sh script to start up a server.

You are looking for a message like this to confirm it is up:

Members {size:1, ver:1} [
    Member [192.168.1.156]:5701 - f4855568-dcd6-4cbc-b231-6f8af5c72487 this
]

This lists the members of the cluster, and if you're starting from clean there should only be one member, the process just started.

Take a note of the process id of this server, for instance by looking in the hazelcast_instance.pid file.

NOTE: The default port for Hazelcast is 5701. 5701 was free, so 5701 was selected.

Write Some Data Using Go

Run:

go run two.go

from the hello-world folder in the Go source hierarchy.

Amongst the output, you should see the same message as before:

Map 'greetings' Size before 0
Map 'greetings' Size after 5

This is confirming that the "greetings" map has changed from empty to having 5 data records.

Start a Second Hazelcast Server

In Hazelcast's bin folder, remove the hazelcast_instance.pid file.

This file is what the start.sh script uses to determine if a process is running, so by removing it we can trick the start.sh script (and lose the ability for the stop.sh script to find it to shut it down cleanly).

So now if we run start.sh, it should start another Hazelcast server.

You should now see messages like this:

Members {size:2, ver:2} [
    Member [192.168.1.156]:5701 - f4855568-dcd6-4cbc-b231-6f8af5c72487
    Member [192.168.1.156]:5702 - 3f84bc0e-5ce2-457f-8db2-c52c82e34d82 this
]

Two Hazelcast servers are running, they have found each other and joined together to form a cluster. Each will log out the presence of the others.

NOTE: The default port for Hazelcast is 5701. 5701 is in use by the first server, so the second server takes the next available one, 5702.

Kill the First Hazelcast Server

Using your OS ( kill -9taskkill, etc.), forcibly terminate the first Hazelcast server that you started.

The second Hazelcast server will notice the first disappear and moan about it.

It should then produce a message with the new member list of the cluster, only containing itself.

Members {size:1, ver:3} [
    Member [192.168.1.156]:5702 - 3f84bc0e-5ce2-457f-8db2-c52c82e34d82 this
]

The proof you've killed the right one of the two is that the one using port 5702 is still running and reporting the other as AWOL.

Read Some Data From Go

From the hello-world folder in your Go source tree, run the third program to inspect the data in Hazelcast.

go run three.go

Problems!

The Go client routine is looking for a Hazelcast server on 127.0.0.1:5701 and there is no server there as we've just killed it.

The Go routine three.go has been deliberately configured with only one place to look for a connection and there's nothing there. That's not good practice.

So it'll produce a slew of error messages starting with:

2017/11/22 20:58:13 the following error occured while trying to connect to cluster:  target is disconnected
2017/11/22 20:58:17 the following error occured while trying to connect to cluster:  target is disconnected

And then it'll shut down.

Start Another Hazelcast Server

Back to Hazelcast's bin folder, remove the hazelcast_instance.pid file again, and run the start.sh again.

A Hazelcast server will start up, find the other one that is still running, and both will produce a message showing who is now in the cluster.

Members {size:2, ver:4} [
    Member [192.168.1.156]:5702 - 3f84bc0e-5ce2-457f-8db2-c52c82e34d82
    Member [192.168.1.156]:5701 - b1536a80-78c6-401e-b870-8515acbbabe7 this
]

Try Again to Read Some Data From Go

Now try the below command again:

go run three.go

Our output should be:

 -> 'Spanish'=='hola mundo'
 -> 'German'=='hallo welt'
 -> 'Italian'=='ciao mondo'
 -> 'English'=='hello world'
 -> 'French'=='bonjour monde'
[5 records]

The five data records are in an unspecified order.

Recap

So what we've seen in the bonus section is some of the power of Hazelcast as a resilient and scalable store.

Although some of the Hazelcast servers may suffer mishaps and go offline, so long as sufficient others remain you don't lose data.

(And you should always be able to find the cluster as it scales up and down if you've configured correctly!)

Summary

Hazelcast has many more features too: queues, topics, executors, entry processors, listeners, etc., etc... a subject for another day.

Be sure to check out the webinar  "Golang for Hazelcast IMDG" to learn more!

Learn how to build distributed stream processing applications in Java that elastically scale to meet demand- includes reference application.

Topics:
web dev ,go ,golang ,hazelcast imdg

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}