Ultra fast Reliable Messaging in Java

Many times in our systems we've to use some messaging platform to provide communication between different servers. Usually we want this platform to be fast (the more the better) and reliable. There are many popular solutions like RabbitMQ, HornetQ and commercial products. But I wanted to try something completely different and really fast so I choose Java-Chronicle! Following Peter Lawrey words: "This library is an ultra low latency, high throughput, persisted, messaging and event driven in memory database. The typical latency is as low as 80 nano-seconds and supports throughput of 5-20 million messages/record updates per second." I will add that it can also synchronously persist it into disk and replicate over network - nice :)

After cloning project from GitHub we can find two major in this context classes: ChronicleSource and ChronicleSink. The first will be our master server, which will be used as endpoint for getting new excerpts (in this post you can assume that is the same as message). It will use the same datastore which is used by message producer. ChronicleSink will connect to source server and will replicate messages into new datastore, even on remote server as it works over TCP.

Ok, at first we've to implement our producer class:

public class ChronicleProducer {

  private static final int MAX_MESSAGES = 10000000;

  public static void main(String[] args) throws IOException {
    IndexedChronicle chronicle =
        new IndexedChronicle("/tmp/chronicle_in");

    Excerpt excerpt =
        chronicle.createExcerpt();

    System.out.println("TestMessageInTheBottle".length());
    for (int i = 1; i < MAX_MESSAGES + 1; i++) {
      excerpt.startExcerpt(36);
      excerpt.writeLong(System.nanoTime());
      excerpt.writeBytes("TestMessageInTheBottle");
      excerpt.writeInt(i);
      excerpt.writeBoolean(i == MAX_MESSAGES);
      excerpt.finish();
    }

    chronicle.close();
  }
}

Then we need something to consume our messages:

public class ChronicleConsumer {

  public static void main(String[] args) throws IOException {
    IndexedChronicle chronicle =
        new IndexedChronicle("/tmp/chronicle_out");

    Excerpt excerpt = chronicle.createExcerpt();

    while (true) {
      if (excerpt.nextIndex()) {
        long timestamp = excerpt.readLong();
        String message = excerpt.readByteString();
        int index = excerpt.readInt();
        System.out.println(index + " message: "
            + message + " created at timestamp " + timestamp);
        if (excerpt.readBoolean()) {
          break;
        }
      } else {
        LockSupport.parkNanos(TimeUnit.MICROSECONDS.toNanos(1));
      }
    }

    chronicle.close();
  }
}

Now we can start up all service!

1. $ java com.higherfrequencytrading.chronicle.tcp.ChronicleSource /tmp/chronicle_in 8099
2. $ java com.higherfrequencytrading.chronicle.tcp.ChronicleSink /tmp/chronicle_out localhost 8099
3. $ java ChronicleConsumer
4. $ java ChronicleProducer