Over a million developers have joined DZone.

Akka Persistence With Apache Ignite

DZone's Guide to

Akka Persistence With Apache Ignite

This new project aims to reap the benefits of both event sourcing and the data grid based on the Journal Plugin and TCK specs provided by Akka persistence.

· Big Data Zone ·
Free Resource

The open source HPCC Systems platform is a proven, easy to use solution for managing data at scale. Visit our Easy Guide to learn more about this completely free platform, test drive some code in the online Playground, and get started today.

In this post, we will share a new project to use Apache Ignite data grid as an event and snapshot store to reap the benefits of both event sourcing and the data grid.

The implementation is based on the Journal Plugin and TCK specs provided by Akka persistence.

It mainly uses Apache Ignite with Akka persistence to provide Journal and Snapshot stores by using the partitioned caches to benefit from distributed, highly available data grid features. Additionally, the nice query and data computation features in Ignite can be used to have normalized views from the event store and do analytical jobs over them. Note that it is advised to keep write nodes separate from read nodes for better scalability.


We are using Akka version 2.5.7+ and Ignite version 2.3.0+.

Journal Plugin

  • All operations required by the Akka persistence Journal Plugin API are fully supported.
  • It uses an Apache Ignite partitioned cache with the default number of backups to 1 that can be changed into a reference.conf file.

Snapshot Store Plugin

How to Use

Enable the plugins in your Akka cluster configuration:

akka.persistence.journal.plugin = "akka.persistence.journal.ignite"
akka.persistence.snapshot-store.plugin = "akka.persistence.snapshot.ignite"

Configure Ignite data grid properties (default configured onlocalhost):

ignite {
  //to start client or server node to connect to Ignite data cluster 
  isClientNode = false
  // for ONLY testing we use localhost
  // used for grid cluster connectivity
  tcpDiscoveryAddresses = "localhost"
  metricsLogFrequency = 0
  // thread pools used by Ignite , should based into target machine specs
  queryThreadPoolSize = 4
  dataStreamerThreadPoolSize = 1
  managementThreadPoolSize = 2
  publicThreadPoolSize = 4
  systemThreadPoolSize = 2
  rebalanceThreadPoolSize = 1
  asyncCallbackPoolSize = 4
  peerClassLoadingEnabled = false
  // to enable or disable durable memory persistance
  enableFilePersistence = true
  // used for grid cluster connectivity, change it to suit your configuration 
  igniteConnectorPort = 11211
  // used for grid cluster connectivity , change it to suit your configuration 
  igniteServerPortRange = "47500..47509"
  //durable memory persistance storage file system path , change it to suit your configuration 
  ignitePersistenceFilePath = "./data"

After performing these steps, you will have Ignite enabled as your Journal and Snapshot Plugins. And you can enable it by starting the server node or client based on the configuration above.

Technical Details

For more details, feel free to dive into the code base — it is a small code base for now!


This is a work in progress any contribution would be really helpful. Please check the project on GitHub and give a hand if you have any suggestions!

Managing data at scale doesn’t have to be hard. Find out how the completely free, open source HPCC Systems platform makes it easier to update, easier to program, easier to integrate data, and easier to manage clusters. Download and get started today.

akka ,apache ignite ,event sourcing ,big data ,data grid ,data persistence ,tck ,journal

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}