If you have a hammer everything looks like a nail, right? So we all witnessed that people sometimes try to solve the problem with wrong technology. Heck we probably all did it at one point or another. Common reasons are familiarity with an exiting technology stack we have at hand or perception that some of the features will justify it all. But using any technology in a way it’s not designed for, will lead to all kind of problems and you’ll eventually be forced to do it properly (probably cursing at the project in question because it hasn’t met you wrong expectations).

In this and couple of follow-up posts I’ll try sum up some things we saw in mailing lists, Jiras, etc related to improper usage of messaging systems (ActiveMQ in particular). Hopefully, it will help people that consider using messaging in their architecture, see if it is the right tool for solving their particular problem.

So let’s kick off with one common mistake people make

Using message queue as a database

Messaging systems are built to asynchronously connect multiple systems, by passing messages between them. So everything is designed with that in mind; how to most efficiently pass messages from producers to consumers. This means that messages are expected to be reasonably short-lived, and not stored in a queue.

From time to time we see people trying keep application state in the broker. Put some messages in a queue, than browse them, cherry-pick just some of them, delete others and similar stuff. While most of the messaging systems have some kind of support to do this it’s not what they’re designed to support primarily. Client APIs, internal storage system, client-server contracts, etc. are optimized for entirely different set of tasks.

An example could be a system that wants to keep a single most-recent data as a queue message (timestamped or versioned somehow). So that application can find that message (usually by browsing) and do the house keeping by deleting stale data. People are sometimes inclined to do this as brokers provide high-availability, reconnection logic, can be geographically distributed which is all fine and well. But brokers are a poor choice for maintaining application state.

A workaround is not to keep any state in the broker, of course. Either use a centralized high-availability database or a local copy of data and use messaging system to propagate changes.

So if you find yourself wanting to store some messages in a queue and then later browse them, query them or maintain them, please don’t. Get yourself a database of some kind (relational or not) and manipulate data there. Queues (and topics) are for data that should be consumed as they come and moved from one system to another as fast as possible. They should live in the broker only as long as it takes to consume them or if something has gone wrong and they cannot be consumed. Which should be an exceptional situation rather than what we design for.

If there’s any messaging anti-patterns you observed (or designed yourself – c’mon don’t be ashamed), send them to me. I’ll gladly put them on my list and document them in coming days.