Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Apache Ignite: How to Read Data From Persistent Store

DZone's Guide to

Apache Ignite: How to Read Data From Persistent Store

In this post, author Dmitriy Setrakyan shows you how to load data from a MySQL database into an Ignite distributed cache. Read on and see how it's done.

· Database Zone ·
Free Resource

Databases are better when they can run themselves. CockroachDB is a SQL database that automates scaling and recovery. Check it out here.

Data can be loaded directly from any persistent store into Apache Ignite caches. This example shows you how to load data from a MySQL database into an Ignite distributed cache. Here, I am assuming that you already have Apache Ignite installed on your system. If not, you can go through this tutorial first.

1. Sample PERSON Table

To start with, here is what the PERSON data in my database looks like: 

Image title

2. Model 

Here is a sample Person.java class corresponding to the PERSON table in the database.

public class Person {

    private long id;

    private long orgId;

    private String name;

    private int salary;

    // Constructor
    …
    // Getter and Setter methods
    …
}

3. Maven Dependency

I have specified the following dependencies in my project’s pom.xml

<dependency>
    <groupid>org.apache.ignite</groupid>
    <artifactid>ignite-core</artifactid>
    <version>1.5.0.final</version>
</dependency>

<dependency>
    <groupid>org.apache.ignite</groupid>
    <artifactid>ignite-spring</artifactid>
    <version>1.5.0.final</version>
</dependency>

<dependency>
    <groupid>mysql</groupid>
    <artifactid>mysql-connector-java</artifactid>
    <version>5.1.6</version>
</dependency>

4. Configure Read-Through

To load the data from the database, you need to enable the read-through mode and set the cacheStoreFactory property of CacheConfiguration. You can set these values either in your Spring XML configuration file or programmatically.

<bean class="org.springframework.jdbc.datasource.DriverManagerDataSource" id="dataSource">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"></property>
    <property name="url" value="jdbc:mysql://localhost:3306/mydbname"></property>
    <property name="username" value="username"></property>
    <property name="password" value="passwd"></property>
</bean>

<bean class="org.apache.ignite.configuration.IgniteConfiguration" id="ignite.cfg">
    <property name="cacheConfiguration">
        <list>
            <bean class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="name" value="personCache"></property>
                <!-- Enable readThrough-->
                <property name="readThrough" value="true"></property>
                <property name="writeThrough" value="true"></property>
                <!-- Set cacheStoreFactory-->
                <property name="cacheStoreFactory">
                    <bean class="javax.cache.configuration.FactoryBuilder" factory-method="factoryOf">
                        <constructor-arg value="myexamples.store.PersonStore"></constructor-arg>
                    </bean>
                </property>
                <property name="queryEntities">
                    <list>
                        <bean class="org.apache.ignite.cache.QueryEntity">
                            <property name="keyType" value="java.lang.Long"></property>
                            <property name="valueType" value="ignite.myexamples.model.Person"></property>
                            <property name="fields">
                                <map>
                                    <entry key="id" value="java.lang.Long"></entry>
                                    <entry key="name" value="java.lang.String"></entry>
                                    <entry key="orgId" value="java.lang.Long"></entry>
                                    <entry key="salary" value="java.lang.Integer"></entry>
                                </map>
                            </property>
                        </bean>
                    </list>
                </property>
            </bean>
        </list>
    </property>

    <property name="peerClassLoadingEnabled" value="true"></property>

    <!-- Other Ignite configurations-->
    ...

</bean>

5. Implement CacheStore

Now that we have our model, maven dependencies, and cache configurations in place, it’s time to implement the store. To load the data from the database, loadCache() and load() methods of the CacheStore interface should be implemented. 

public class PersonStore implements CacheStore<Long, Person> {

    @SpringResource(resourceName = "dataSource")
    private DataSource dataSource;

    // This method is called whenever IgniteCache.loadCache() method is called.
    @Override
    public void loadCache(IgniteBiInClosure<Long, Person> clo, @Nullable Object... objects) throws CacheLoaderException {
        System.out.println(">> Loading cache from store...");

        try (Connection conn = dataSource.getConnection()) {
            try (PreparedStatement st = conn.prepareStatement("select * from PERSON")) {
                try (ResultSet rs = st.executeQuery()) {
                    while (rs.next()) {
                        Person person = new Person(rs.getLong(1), rs.getLong(2), rs.getString(3), rs.getInt(4));

                        clo.apply(person.getId(), person);
                    }
                }
            }
        }
        catch (SQLException e) {
            throw new CacheLoaderException("Failed to load values from cache store.", e);
        }
    }

    // This method is called whenever IgniteCache.get() method is called.
    @Override
    public Person load(Long key) throws CacheLoaderException {
        System.out.println(">> Loading person from store...");

        try (Connection conn = dataSource.getConnection()) {
            try (PreparedStatement st = conn.prepareStatement("select * from PERSON where id = ?")) {
                st.setString(1, key.toString());

                ResultSet rs = st.executeQuery();

                return rs.next() ? new Person(rs.getLong(1), rs.getLong(2), rs.getString(3), rs.getInt(4)) : null;
            }
        }
        catch (SQLException e) {
            throw new CacheLoaderException("Failed to load values from cache store.", e);
        }
    }

    // Other CacheStore method implementations.
    …
}

For convenience purposes, Ignite also provides it users with CacheStoreAdapter class that has a default implementation for some of the CacheStore methods - loadAll()writeAll(), and deleteAll().

6. Load Cache

Here is a sample PersonStoreExample.java class that makes a call to IgniteCache.loadCache()  method which internally delegates the call to CacheStore.loadCache() method (which we implemented in the previous step).

public class PersonStoreExample {
    public static void main(String[] args) throws IgniteException {
        Ignition.setClientMode(true);

        try (Ignite ignite = Ignition.start("config/cluster-config.xml")) {
            try (IgniteCache<Long, Person> cache = ignite.getOrCreateCache("personCache")) {
                // Load cache with data from the database.
                cache.loadCache(null);

                // Execute query on cache.
                QueryCursor<List<?>> cursor = cache.query(new SqlFieldsQuery(
                        "select id, name from Person"));

                System.out.println(cursor.getAll());
            }
        }
    }
}

7. Start Ignite Cluster

From the command shell, lead yourself to the Ignite installation folder and start the server nodes, using the following command:

$ bin/ignite.sh <path-to-Spring-XML-configuration-file>

Make sure that PersonStore.java is in the class path of Ignite. To do so, you can set the USER_LIBS environment variable, or drop the project jar into the libs folder of your Ignite installation. 

8. Output

From your IDE, run PersonStoreExample.java.

Image title


For more information, documentation, and screencasts visit the Apache Ignite website.

Databases should be easy to deploy, easy to use, and easy to scale. If you agree, you should check out CockroachDB, a scalable SQL database built for businesses of every size. Check it out here. 

Topics:
caching ,apache ignite ,tutorial ,database ,data grid ,compute grid ,sql

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}