Introduction to In-Memory Data Grid: Main Features

It is not a new attempt at all to use main memory as a storage area instead of a disk. You can find in your daily life many cases in which Main Memory DBMS (MMDB) is used to execute much faster than a disk. One example is when you use a mobile phone. When you SMS or call your friend, most mobile service providers use MMDB to get the information on your friend as soon as possible. 

In Memory Data Grid (IMDG) is the same as MMDB in that it stores data in main memory, but it has a totally different architecture. The features of IMDG can be summarized as follows: 

• Data is distributed and stored in multiple servers.
• Each server operates in the active mode.
• A data model is usually object-oriented (serialized) and non-relational.
• According to the necessity, you often need to add or reduce servers.

In other words, IMDG is designed to store data in main memory, ensure scalability and store an object itself. IMDG products, whether open source or commercial product, include:  

• Hazelcast
• Terracotta Enterprise Suite
• VMware Gemfire
• Oracle Coherence
• Gigaspaces XAP Elastic Caching Edition
• IBM eXtreme Scale
• JBoss Infinispan

This article does not aim to compare the features and performance of these products. Instead we will look into the architecture of IMDG and discuss how NHN can utilize it.

Why Memory?

As of June 2012, an SSD using SATA interface performs approximately 500 MB/s, while an SSD using expensive PCI Express records approximately 3,000 MB/s. As the performance of 10,000 RPM SATA HDD is approximately 150 MB/s, an SSD is 4-20 times faster than an HDD. However, the performance of DDR3-2500 reaches 20,000 MB/s. The processing performance of main memory is 800 times faster than an HDD, 40 times faster than an SSD and seven times faster than the fastest SDD. Moreover, the latest x86 server supports main memory of hundreds of GB per server.

Michael Stonebraker says the typical Online Transaction Processing (OLTP) data capacity is approximately 1 TB and that the OLTP processing data capacity would not increment well. If servers using main memory of 1 TB or larger become more commonly used, you will be able to conduct operations with the entire data placed in main memory, at least in the field of OLTP.

In the history of computing, "Make it faster" has always been the best virtue everyone should pursue. As the capacity of main memory incremented, it was necessary for platforms actively using main memory as a storage area instead of permanent storage to appear.

IMDG Architecture

To use main memory as a storage area, you should overcome two weak points: limited capacity andreliability. You need to handle data that exceeds the maximum capacity of the main memory of the server and also make no data loss take place when a failure occurs. IMDG overcomes the limit of capacity by ensuring horizontal scalability using a distributed architecture, and resolves the issue of reliability through a replication system.

Despite some differences in details for each product, you can generalize an IMDG architecture as in Figure 1:

Figure 1: IMDG Architecture.

An application server has a client library provided by IMDG and it accesses IMDG by using this library. Many IMDG products provide the feature of synchronizing data to RDBMS. However, you don't need to necessarily establish a separate permanent storage system (e.g., RDBMS). In general, IMDB enables objects to be stored through serialization. Some products provide the feature of storing objects that implement serializable interface, while some IMDGs provide an independent serialization method. Of course, it is very convenient to use them thanks to the schemaless structure. It can be understood as the concept of In Memory Key-Value Database to store and retrieve objects.

The data model used in IMDG is Key-Value. Therefore, data can be distributed and stored by using this key. There are a variety of methods ranging from using a consistency hash model as in Arcus (NHN's Memcached Cloud) to using a simple modulo method as in Hazelcast. When storing data in this way, at least one node is used as a replication system to respond to any failure.

There are a variety of interfaces by products. Some products provide an SQL-like syntax for access via JDBC, and some provide API implementing Java's Collection where you can use HashMap or HashSet for multiple nodes.

IMDG has a different usage and purpose compared to cache systems such as Arcus. Figure 2 shows a simplified structure of Arcus Architecture.

Figure 2: Cache System Architecture Such as Arcus.

Cache systems such as Arcus also use main memory as a storage area and secure horizontal scalability. In this sense, they are the same as IMDG. However, the use method and purposes shown in Figure 1 and 2 are significantly different. It is necessary to use a permanent storage area in cache systems such as Arcus, but it is optional in IMDG.

The cache system and IMDG also differ in the respect of whether data migration is available, whether reliability is ensured and whether a replication feature is provided.

Features of IMDG

A list of the features of HazelCast, one of IMDG products, is as follows. As HazelCast holds a double license policy, you must purchase a commercial license to use some features, such as ElasticMemory. You can find use reference information easily because many features of the product can be used freely as open source.

Although it is difficult to say the features of HazelCast are the common ones provided by all the other IMDG products, I decided to introduce it here because I considered HazelCast a good example to understand the features of IMDG.

DistributedMap & DistributedMultiMap

It is a class which implements Map<?, ?>. Map data is distributed and allocated to multiple IMDG nodes. As a table of RDBMS can be expressed as Map<Object key, List<Object>>, you can get the data distribution effect similar to when using sharded RDBMS.

HazelCast also supports SQL-like features in DistributedMap. When inspecting values in a map, you can use SQL-like clauses, such as WHERE clause, LIKE, IN and BETWEEN.

HazelCast provides not only the feature of storing all data in memory but also provides the feature of storing it in permanent storage. When storing data in permanent storage, you can configure it to be used as a cache system. You can also store only necessary data in memory and put the remaining less frequently used data in permanent storage by selecting the LRU or LFU algorithms.

In addition, you can use MultiMap in a distributed environment. If you retrieve a certain key, you can get a value list in the form of Collection<Object>.

Distributed Collections

You can use DistributedSet, DistributedList or DistributedQueue. Data in such a Distributed Collection object is stored not in a single IMDG node, but distributed and stored in multiple nodes. For this reason, it is possible to maintain a single list object or set object stored in multiple nodes. 

DistributedTopic & DistributedEvent

HazelCast provides the feature of topic reading that guarantees the order of messages being published. This means you can use it as a distributed message queue system. 

DistributedLock

This is literally a distributed lock. You can conduct a synchronization in multiple distributed systems by using a single lock. 

Transactions

You can use transactions for DistributedMap and DistributedQueue. With the feature of commit/rollback, you can use IMDG even in an environment where operations should be executed more carefully.

Using Large Capacity Memory and GC 

Most of the products introduced above use Java as an implementation language. As a heap of tens of GB should be used, it could take quite a long time to complete a full GC. For this reason, IMDG provides a method to overcome this limit. The method is to use an off-heap memory (direct buffer). When JVM receives a direct buffer creation request, it allocates memory to the space out of a heap and uses it. Objects are stored in this allocated space. Because a direct buffer is not a space for GC any more, no full GC problem will occur. In general, accessing a direct buffer takes longer than it takes to access a heap buffer. Nevertheless, if you use a direct buffer, you can allocate large space and reduce the burden of a full GC. Therefore you can have the advantage to secure a certain amount of processing time by eliminating the time for a full GC when using large capacity memory space.

Figure 3: Comparison of a Heap, a Direct Buffer and a Disk (Source).

However, it requires expertise to store and to retrieve objects by using a direct buffer. You need to have the expertise required to make a memory allocator. For this reason, this feature of storing objects by using off-heap memory is provided only in commercial IMDG products.

If Integrity Constraint is Provided

Currently IMDG is used mainly as a cache system. However, an IMDG is a platform that is likely to develop into a main storage area. In many cases, a distributed map can reliably replace a table of RDBMS. Some products provide a distributed lock and if the integrity constraint feature can be provided based on such distributed lock, it is possible to replace RDBMS with IMDG. In this case, you will be able to respond to statistics processing by using RDBMS as a back-end system. This means the role of RDBMS in Internet services can become assistive. If the integrity constraint feature is provided, it is possible to provide a pleasant user experience based on fast speed as well as other features that were difficult to provide in the past.

I think, given such possibilities, it is necessary to seriously examine the values of using an IMDG as the main storage in NHN Internet services.