ActiveMQ and the JMSRedelivered Header

This post applies to ActiveMQ version 5.9.0 and below. ActiveMQ 5.9.0 is the latest released version at the time of writing this post. It also applies to JBoss A-MQ 6.0 and below.

The JMSRedelivered header is set on a JMS message to indicate that a redelivery of this message is occurring. Or to use the quote from the JMS 1.1. spec:

"If a client receives a message with the JMSRedelivered indicator set, it is likely, but not guaranteed, that this message was delivered but not acknowledged in the past."

In ActiveMQ messages can be prefetched to consumers. Let say a consumer gets 1000 messages prefetched but for some reason disconnects after consuming and acknowledging 900 messages. When the broker looses the connection to the consumer, it knows that out of the 1000 prefetched messages only 900 were acknowledged, so the remaining 100 messages will get the JMSRedelivered flag set to true to indicate that these messages get redelivered.
Another or the same consumer reconnecting can examine this flag and perhaps process the message differently (e.g. run extra duplicate detection to verify this message has not been processed before).

If however your broker crashes or shuts down before redelivering the 100 messages that have JMSRedelivered=true set, then this redelivery flag will be lost by default. The reason is that ActiveMQ stores a message to persistent storage (e.g. KahaDB) only when it initially receives the message from the producer. It does not write the message to storage thereafter. So the JMSRedelivered flag is set on the JMS message that is held in memory only. After a restart of the broker none of the remaining 100 messages would have JMSRedelivered=true.

For the KahaDB persistence adapter there has been an improvement introduced in ActiveMQ 5.6 as part of AMQ-3519. With this improvement it is possible to configure KahaDB to persist the change to the JMSRedelivered flag for a message in the persistence store, if the consumer is transacted. In case the consumer rolls back the transaction, the broker will re-write the message to the store with an updated redelivery count. This obviously has a performance impact, which should generally be rather low, assuming that most transactions will commit and rollbacks occur only occasionally.

This behavior is not enabled out of the box. To enable it you need to
- set transactedIndividualAck=true on the ConnectionFactory, and
- add rewriteOnRedelivery=true property to the KahaDB persistence store configuration

Again this feature only applies to using the KahaDB persistence store and transacted consumers. Non-transacted consumers cannot benefit from it. Also, if the broker crashes during such rollback (before it rewrites the messages with an updated redelivery count to KahaDB), the broker will also loose the JMSRedelivered information.

Now with AMQ-5068 we raised another improvement. Rather than the JMSRedelivered header only working for transacted consumers, it should work for every consumer, transacted and non-transacted. And it should also remain consistent if the broker crashes.

This feature (when it gets implemented) will obviously not be enabled by default either. As every single JMS message needs to be rewritten to the store with an updated redelivery count prior to dispatching it to a consumer, this will have a significant performance overhead. We essentially write each message twice to the store (when we receive it from the producer but before we ack the message to the producer and before we dispatch it to the consumer). Writing to the store requires a sync write which typically has a very high cost on performance.

Apparently WebSphere MQ has a similar feature when setting the "Harden get backout" attribute to HARDENBO. See this WebSphere MQ doc. They also issue a warning that this will drastically effect the message throughput.