Over a million developers have joined DZone.

Deploying a RabbitMQ Cluster at One Million Messages Per Second on Google Compute Engine

DZone's Guide to

Deploying a RabbitMQ Cluster at One Million Messages Per Second on Google Compute Engine

· Integration Zone ·
Free Resource

How to Transform Your Business in the Digital Age: Learn how organizations are re-architecting their integration strategy with data-driven app integration for true digital transformation.

Message queuing systems are an integral part of many business models where scalable applications are at the heart of systems development. One of the most popular (and growing) message queuing systems is RabbitMQ. I know that a lot of DZone developers have backed it as a great choice for a high-volume low-latency MQ, and I've seen it grown in popularity even in the last year.

Last week, Google and Pivotal released a guide to getting started with RabbitMQ clusters on Google Compute Engine. Google has backed RabbitMQ in this regard due to what I mentioned previously--high through-put, low latency, and in-memory queuing--all in a way that is easy to set up and deploy. Now Google and Pivotal are backing up their RabbitMQ praise with a benchmark release that demonstrates a RabbitMQ cluster handling a million messages per second through Google Compute Engine--a fantastic feat that definitely seems to back up their claims about high-performance on their platform. They spoke a little bit about their motivations:

"We also wanted to demonstrate that a RabbitMQ cluster running on Google Compute Engine is a good fit for high-throughput and low-latency scenarios. To do so, we ran a series of benchmarks in various configurations. These test clusters were able to sustain a throughput of over 1 million messages published and consumed per second (a sustained combined ingress/egress of over two million messages per second)."
So the idea is that Google rounded up the engineers at Pivotal (the company behind RabbitMQ). For their Google Compute Engine platform setup, they provisioned a cluster of 32 virtual machines with 8 vCPUs and 30GB of RAM each, and deployed RabbitMQ on Debian. To generate the load they used the RabbitMQ PerfTest tool. Here's some evidence of their benchmarks:

Screen Shot 2014-04-23 at 2.04.02 PM.png

These are definitely results that should interest the community at large. They reached a final conclusion:

We are excited to offer our customers an easy way to deploy RabbitMQ. We are proud that both Google and Pivotal benchmarks validate that Compute Engine is an excellent choice for running RabbitMQ in a public cloud

To deploy your own dedicated RabbitMQ cluster, go to Google Developer Console and navigate to an existing project, or create a new one, and you will have an option to deploy RabbitMQ from the project dashboard

Build and deploy API integrations 7x faster. Try the Cloud Elements 100% RESTful platform for 30 days free. Access your trial here.


Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}