Introducing ElasticMQ: Scala SQS Alternative

Wanting to explore Scala and some new Scala/Java libraries I started writing ElasticMQ. It is a simple message queueing system, following closely Amazon SQS semantics and exposing a REST SQS-like interface. Currently only the basic operations are implemented. ElasticMQ can be useful as an SQS replacement, e.g. for for testing SQS applications.

My aim was to make the usage as simple as possible, that’s why to create a new node and expose the REST interface all you need to do is:

val node = NodeBuilder.createNode
val server = SQSRestServerFactory.start(node.nativeClient, 8888,
                 "http://localhost:8888")

Now simply point your SQS client to http://localhost:8888 and you should be able to create/delete queues, change the visibility timeout, send and receive messages.
When you are done using ElasticMQ, you can shutdown the node and the server:

server.stop()
node.shutdown()

All of the source code is available on GitHub. See the readme for instructions on how to add ElasticMQ as a dependency to your Maven or SBT project.

On the technical side, ElasticMQ uses a number of interesting things.

Firstly, there’s Netty, an asynchronous, event-driven java NIO framework. I used Netty to implement the REST server. All of the requests are processed in a non-blocking way.

The messages are currently stored in an in-memory H2 database (but this can be easily changed to point e.g. to a MySQL or PostgreSQL DB), and managed using the Squeryl Scala library. Squeryl lets you write SQL queries as typesafe Scala code, e.g.

update(messages)(m =>
  where(m.id === message.id and m.lastDelivered === message.lastDelivered)
    set(m.lastDelivered := lastDelivered))

For integration testing I am using the Typica library.

On the Scala side, I created a simple internal DSL for defining rest handlers. The server (which uses Netty) is generic and can be provided with any number of “rest handlers”, to which requests are dispatched. Each rest handler specifies what is the path it will respond to, what are the required parameters and what should be run in response to a request. For example:

val handler1 = (createHandler
   forMethod GET
   forPath (root / "products" / %("name") / "price")
   requiringParameters List("currency", "quantity")
   requiringParameterValues Map("department"->"electronics")
   running electronicsPriceRequestLogic)
 
val server = RestServer.start(handler1 :: handler2 :: … :: Nil, 8888)

See the RequestHandlerLogic file for the implementation and RequestHandlerBuilderTestSuite for example usage.

In fact there’s another small DSL for defining paths. For example (root / "a" / "b" / "c") will only match a/b/c, but (root / "a" / %("p1") / "c") will match a/anything/c, with the middle component being assigned to the p1 parameter. You can also use regex, e.g. (root / "a" / """[a-z0-9]+""".r / "c") for matching path components. See the RestPath class and test for details.

Not sure where ElasticMQ will go in the future, but I can image lots of interesting possibilities. So stay tuned :).

Comments about the code, Scala style, Netty/Squeryl usage are very welcome! As well as any other ideas, improvement suggestions etc.

Adam