How Many Queues Are Best For Max Performance? RabbitMQ
The Integration Zone is brought to you in partnership with Red Hat. Download the IDC Report: The Business Value of Red Hat Integration Products to learn more about Red hat Integration.
From the RabbitMQ blog:
RabbitMQ's queues are fastest when they're empty. When a queue is empty, and it has consumers ready to receive messages, then as soon as a message is received by the queue, it goes straight out to the consumer. In the case of a persistent message in a durable queue, yes, it will also go to disk, but that's done in an asynchronous manner and is buffered heavily. The main point is that very little book-keeping needs to be done, very few data structures are modified, and very little additional memory needs allocating.
From the rabbitmq-discuss mailing group:
- Use a larger prefetch count. Small values hurt performance.
- A topic exchange is slower than a direct or a fanout exchange.
- Make sure queues stay short. Longer queues impose more processing overhead.
- If you care about latency and message rates then use smaller messages. Use an efficient format (e.g. avoid XML) or compress the payload.
- Experiment with HiPE, which helps performance.
- Avoid transactions and persistence. Also avoid publishing in immediate or mandatory mode. Avoid HA. Clustering can also impact performance.
- You will achieve better throughput on a multi-core system if you have multiple queues and consumers.
- Use at least v2.8.1, which introduces flow control. Make sure the memory and disk space alarms never trigger.
- Virtualisation can impose a small performance penalty.
- Tune your OS and network stack. Make sure you provide more than enough RAM. Provide fast cores and RAM.
Hope that gives you some ideas for improving your queueing performance.