ActiveMQ: High Performance Messaging Using LevelDB

ActiveMQ is already built for high performance and is one of the fastest brokers available but did you know it can be even faster by configuring the LevelDB message store?  LevelDB is a fast light weight key-value storage system which maps byte keys to byte values.

Not too long ago I came across an issue where the ActiveMQ broker seems to be experiencing a performance issue.  This issue arose from a very specific use case, none the less it has highlighted the exceptional performance of the LevelDB store.  This use case involved durable topic subscribers with selectors; one subscriber with no selector to receive all messages, one subscriber with a selector of "true", and one subscriber with a selector of "false".  A producer was then configure to alternate the property set on the message to "true" and "false".

Using the kahaDB store I saw that it would take upwards of 7 hours for 200,000 messages to be published to the broker.  Out of curiosity I switch to the LevelDB store and saw the exact same scenario only took 70 seconds.  This was absolutely amazing!

If you are interested in enabling the LevelDB store it can be configured with a couple simple steps.  Depending on the version of the broker you are running there are different instructions.

Fuse Distributions

First, download the fuse-leveldb-1.3-uber.jar (which is the latest at the time of this writing) and copy the JAR into the broker's lib directory.  Then, in the broker configuration file (activemq.xml) update the persistence adapter to use the LevelDB store as follows:

 <persistenceAdapter>  
  <bean xmlns="http://www.springframework.org/schema/beans"   
    class="org.fusesource.mq.leveldb.LevelDBStore">  
   <property name="directory" value="${activemq.base}/data/leveldb"/>  
   <property name="logSize" value="107374182"/>  
  </bean>  
 </persistenceAdapter> 

Apache Distributions

The Apache distribution of ActiveMQ 5.7 and greater come with the LevelDB library so all you need to do is enable the persistence adapter in the broker configuration:

 <persistenceAdapter>  
  <levelDB directory="${activemq.base}/data/leveldb" logSize="107374182"/>  
 </persistenceAdapter>  

That's all there is to it.  For more information on the ActiveMQ LevelDB store and performance metrics check out the GitHub project page at FuseMQ-LevelDB.  For a list of all the available binaries for the fuse-leveldb library have a look at this fusesource repository.