DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Integrating PostgreSQL Databases with ANF: Join this workshop to learn how to create a PostgreSQL server using Instaclustr’s managed service

Mobile Database Essentials: Assess data needs, storage requirements, and more when leveraging databases for cloud and edge applications.

Monitoring and Observability for LLMs: Datadog and Google Cloud discuss how to achieve optimal AI model performance.

Automated Testing: The latest on architecture, TDD, and the benefits of AI and low-code tools.

Related

  • Demystifying Project Loom: A Guide to Lightweight Threads in Java
  • Auditing Spring Boot Using JPA, Hibernate, and Spring Data JPA
  • How To Validate Archives and Identify Invalid Documents in Java
  • Unraveling Lombok's Code Design Pitfalls: Exploring Encapsulation Issues

Trending

  • Understanding Europe's Cyber Resilience Act and What It Means for You
  • Development of Custom Web Applications Within SAP Business Technology Platform
  • Edge Data Platforms, Real-Time Services, and Modern Data Trends
  • LLMs for Bad Content Detection: Pros and Cons
  1. DZone
  2. Coding
  3. Java
  4. Ultra fast Reliable Messaging in Java

Ultra fast Reliable Messaging in Java

Jakub Kubrynski user avatar by
Jakub Kubrynski
·
Jun. 13, 13 · Interview
Like (0)
Save
Tweet
Share
17.90K Views

Join the DZone community and get the full member experience.

Join For Free
Many times in our systems we've to use some messaging platform to provide communication between different servers. Usually we want this platform to be fast (the more the better) and reliable. There are many popular solutions like RabbitMQ, HornetQ and commercial products. But I wanted to try something completely different and really fast so I choose Java-Chronicle! Following Peter Lawrey words: "This library is an ultra low latency, high throughput, persisted, messaging and event driven in memory database. The typical latency is as low as 80 nano-seconds and supports throughput of 5-20 million messages/record updates per second." I will add that it can also synchronously persist it into disk and replicate over network - nice :)

After cloning project from GitHub we can find two major in this context classes: ChronicleSource and ChronicleSink. The first will be our master server, which will be used as endpoint for getting new excerpts (in this post you can assume that is the same as message). It will use the same datastore which is used by message producer. ChronicleSink will connect to source server and will replicate messages into new datastore, even on remote server as it works over TCP.

Ok, at first we've to implement our producer class:
public class ChronicleProducer {

  private static final int MAX_MESSAGES = 10000000;

  public static void main(String[] args) throws IOException {
    IndexedChronicle chronicle =
        new IndexedChronicle("/tmp/chronicle_in");

    Excerpt excerpt =
        chronicle.createExcerpt();

    System.out.println("TestMessageInTheBottle".length());
    for (int i = 1; i < MAX_MESSAGES + 1; i++) {
      excerpt.startExcerpt(36);
      excerpt.writeLong(System.nanoTime());
      excerpt.writeBytes("TestMessageInTheBottle");
      excerpt.writeInt(i);
      excerpt.writeBoolean(i == MAX_MESSAGES);
      excerpt.finish();
    }

    chronicle.close();
  }
}

Then we need something to consume our messages:
public class ChronicleConsumer {

  public static void main(String[] args) throws IOException {
    IndexedChronicle chronicle =
        new IndexedChronicle("/tmp/chronicle_out");

    Excerpt excerpt = chronicle.createExcerpt();

    while (true) {
      if (excerpt.nextIndex()) {
        long timestamp = excerpt.readLong();
        String message = excerpt.readByteString();
        int index = excerpt.readInt();
        System.out.println(index + " message: "
            + message + " created at timestamp " + timestamp);
        if (excerpt.readBoolean()) {
          break;
        }
      } else {
        LockSupport.parkNanos(TimeUnit.MICROSECONDS.toNanos(1));
      }
    }

    chronicle.close();
  }
}

Now we can start up all service!
  1. $ java com.higherfrequencytrading.chronicle.tcp.ChronicleSource /tmp/chronicle_in 8099
  2. $ java com.higherfrequencytrading.chronicle.tcp.ChronicleSink /tmp/chronicle_out localhost 8099
  3. $ java ChronicleConsumer
  4. $ java ChronicleProducer
And how does it work? Let's check from the end :) ChronicleConsumer is reading excerpts from /tmp/chronicle_out which is supplied by ChronicleSink. ChronicleSink connects to localhost:8099 and asks for new messages (sending index of the recent message that it received). On localhost:8099 listens ChronicleSource which looks in /tmp/chronicle_in for message requested by Sink. And /tmp/chronicle_in is supplied by ChronicleProducer :) That's all! Extremely easy and ultra fast. Whole cycle (produce -> send -> receive -> consume) takes about 20 seconds for 10 million messages on my i5 with both stores on single ssd drive.
Java (programming language)

Published at DZone with permission of Jakub Kubrynski, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Demystifying Project Loom: A Guide to Lightweight Threads in Java
  • Auditing Spring Boot Using JPA, Hibernate, and Spring Data JPA
  • How To Validate Archives and Identify Invalid Documents in Java
  • Unraveling Lombok's Code Design Pitfalls: Exploring Encapsulation Issues

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: