Apache Ignite: How to Read Data From Persistent Store

Data can be loaded directly from any persistent store into Apache Ignite caches. This example shows you how to load data from a MySQL database into an Ignite distributed cache. Here, I am assuming that you already have Apache Ignite installed on your system. If not, you can go through this tutorial first.

1. Sample PERSON Table

To start with, here is what the PERSON data in my database looks like: 

2. Model 

Here is a sample Person.java class corresponding to the PERSON table in the database.

public class Person {

    private long id;

    private long orgId;

    private String name;

    private int salary;

    // Constructor
    …
    // Getter and Setter methods
    …
}

3. Maven Dependency

I have specified the following dependencies in my project’s pom.xml: 

<dependency>
    <groupid>org.apache.ignite</groupid>
    <artifactid>ignite-core</artifactid>
    <version>1.5.0.final</version>
</dependency>

<dependency>
    <groupid>org.apache.ignite</groupid>
    <artifactid>ignite-spring</artifactid>
    <version>1.5.0.final</version>
</dependency>

<dependency>
    <groupid>mysql</groupid>
    <artifactid>mysql-connector-java</artifactid>
    <version>5.1.6</version>
</dependency>

4. Configure Read-Through

To load the data from the database, you need to enable the read-through mode and set the cacheStoreFactory property of CacheConfiguration. You can set these values either in your Spring XML configuration file or programmatically.

<bean class="org.springframework.jdbc.datasource.DriverManagerDataSource" id="dataSource">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"></property>
    <property name="url" value="jdbc:mysql://localhost:3306/mydbname"></property>
    <property name="username" value="username"></property>
    <property name="password" value="passwd"></property>
</bean>

<bean class="org.apache.ignite.configuration.IgniteConfiguration" id="ignite.cfg">
    <property name="cacheConfiguration">
        <list>
            <bean class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="name" value="personCache"></property>
                <!-- Enable readThrough-->
                <property name="readThrough" value="true"></property>
                <property name="writeThrough" value="true"></property>
                <!-- Set cacheStoreFactory-->
                <property name="cacheStoreFactory">
                    <bean class="javax.cache.configuration.FactoryBuilder" factory-method="factoryOf">
                        <constructor-arg value="myexamples.store.PersonStore"></constructor-arg>
                    </bean>
                </property>
                <property name="queryEntities">
                    <list>
                        <bean class="org.apache.ignite.cache.QueryEntity">
                            <property name="keyType" value="java.lang.Long"></property>
                            <property name="valueType" value="ignite.myexamples.model.Person"></property>
                            <property name="fields">
                                <map>
                                    <entry key="id" value="java.lang.Long"></entry>
                                    <entry key="name" value="java.lang.String"></entry>
                                    <entry key="orgId" value="java.lang.Long"></entry>
                                    <entry key="salary" value="java.lang.Integer"></entry>
                                </map>
                            </property>
                        </bean>
                    </list>
                </property>
            </bean>
        </list>
    </property>

    <property name="peerClassLoadingEnabled" value="true"></property>

    <!-- Other Ignite configurations-->
    ...

</bean>

5. Implement CacheStore

Now that we have our model, maven dependencies, and cache configurations in place, it’s time to implement the store. To load the data from the database, loadCache() and load() methods of the CacheStore interface should be implemented. 

public class PersonStore implements CacheStore<Long, Person> {

    @SpringResource(resourceName = "dataSource")
    private DataSource dataSource;

    // This method is called whenever IgniteCache.loadCache() method is called.
    @Override
    public void loadCache(IgniteBiInClosure<Long, Person> clo, @Nullable Object... objects) throws CacheLoaderException {
        System.out.println(">> Loading cache from store...");

        try (Connection conn = dataSource.getConnection()) {
            try (PreparedStatement st = conn.prepareStatement("select * from PERSON")) {
                try (ResultSet rs = st.executeQuery()) {
                    while (rs.next()) {
                        Person person = new Person(rs.getLong(1), rs.getLong(2), rs.getString(3), rs.getInt(4));

                        clo.apply(person.getId(), person);
                    }
                }
            }
        }
        catch (SQLException e) {
            throw new CacheLoaderException("Failed to load values from cache store.", e);
        }
    }

    // This method is called whenever IgniteCache.get() method is called.
    @Override
    public Person load(Long key) throws CacheLoaderException {
        System.out.println(">> Loading person from store...");

        try (Connection conn = dataSource.getConnection()) {
            try (PreparedStatement st = conn.prepareStatement("select * from PERSON where id = ?")) {
                st.setString(1, key.toString());

                ResultSet rs = st.executeQuery();

                return rs.next() ? new Person(rs.getLong(1), rs.getLong(2), rs.getString(3), rs.getInt(4)) : null;
            }
        }
        catch (SQLException e) {
            throw new CacheLoaderException("Failed to load values from cache store.", e);
        }
    }

    // Other CacheStore method implementations.
    …
}

For convenience purposes, Ignite also provides it users with CacheStoreAdapter class that has a default implementation for some of the CacheStore methods - loadAll(), writeAll(), and deleteAll().

6. Load Cache

Here is a sample PersonStoreExample.java class that makes a call to IgniteCache.loadCache()  method which internally delegates the call to CacheStore.loadCache() method (which we implemented in the previous step).

public class PersonStoreExample {
    public static void main(String[] args) throws IgniteException {
        Ignition.setClientMode(true);

        try (Ignite ignite = Ignition.start("config/cluster-config.xml")) {
            try (IgniteCache<Long, Person> cache = ignite.getOrCreateCache("personCache")) {
                // Load cache with data from the database.
                cache.loadCache(null);

                // Execute query on cache.
                QueryCursor<List<?>> cursor = cache.query(new SqlFieldsQuery(
                        "select id, name from Person"));

                System.out.println(cursor.getAll());
            }
        }
    }
}

7. Start Ignite Cluster

From the command shell, lead yourself to the Ignite installation folder and start the server nodes, using the following command:

$ bin/ignite.sh <path-to-Spring-XML-configuration-file>

Make sure that PersonStore.java is in the class path of Ignite. To do so, you can set the USER_LIBS environment variable, or drop the project jar into the libs folder of your Ignite installation. 

8. Output

From your IDE, run PersonStoreExample.java.

For more information, documentation, and screencasts visit the Apache Ignite website.