We use Storm for distributed processing. We've been using JMS as a means of driving work into Storm via the spout we developed:
https://github.com/hmsonline/storm-jms
But since Storm uses Zookeeper under the hood, we thought we could
perhaps use Zookeeper instead of JMS for certain low-throughput
distributed messaging needs, thereby decreasing the number of
technologies we needed to maintain in our stack for certain
applications.
Fortunately, there is a Distributed Queue recipe for Zookeeper.
Even better, Netflix has bundled an implementation of this recipe into Curator.
The docs for the Curator recipe were adequate, but I didn't see a
complete example. After some muddling around, I was able to get it
working. You can find the code here:
https://github.com/boneill42/zookeeper-distributed-queue-example
In the end, the throughput was just too slow. (You could see the messages scroll by on the screen as they were being consumed)
Eventually, I punted and decided to take a look at Kafka. That
experiment is going *extremely* well. Kafka is very impressive. More
on that in a future post. =)
Join the DZone community and get the full member experience.
Join For FreeEasily build powerful user management, authentication, and authorization into your web and mobile applications. Download this Forrester report on the new landscape of Customer Identity and Access Management, brought to you in partnership with Stormpath.
The Java Zone is brought to you by Stormpath—a complete, pre-built User Management API. Want to learn how to use JWTs to protect microservices from CSRF and more? Check out this on-demand webinar with our Java Developer Evangelist, Micah Silverman.
Topics:
Published at DZone with permission of Brian O' Neill, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
{{ parent.title || parent.header.title}}
{{ parent.tldr }}
{{ parent.linkDescription }}
{{ parent.urlSource.name }}