Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Apache Ignite: How to Read Data From Persistent Store

DZone's Guide to

Apache Ignite: How to Read Data From Persistent Store

In this post, author Dmitriy Setrakyan shows you how to load data from a MySQL database into an Ignite distributed cache. Read on and see how it's done.

· Database Zone
Free Resource

Check out the IT Market Clock report for recommendations on how to consolidate and replace legacy databases. Brought to you in partnership with MariaDB.

Data can be loaded directly from any persistent store into Apache Ignite caches. This example shows you how to load data from a MySQL database into an Ignite distributed cache. Here, I am assuming that you already have Apache Ignite installed on your system. If not, you can go through this tutorial first.

1. Sample PERSON Table

To start with, here is what the PERSON data in my database looks like: 

Image title

2. Model 

Here is a sample Person.java class corresponding to the PERSON table in the database.

public class Person {

    private long id;

    private long orgId;

    private String name;

    private int salary;

    // Constructor
    …
    // Getter and Setter methods
    …
}

3. Maven Dependency

I have specified the following dependencies in my project’s pom.xml

<dependency>
    <groupid>org.apache.ignite</groupid>
    <artifactid>ignite-core</artifactid>
    <version>1.5.0.final</version>
</dependency>

<dependency>
    <groupid>org.apache.ignite</groupid>
    <artifactid>ignite-spring</artifactid>
    <version>1.5.0.final</version>
</dependency>

<dependency>
    <groupid>mysql</groupid>
    <artifactid>mysql-connector-java</artifactid>
    <version>5.1.6</version>
</dependency>

4. Configure Read-Through

To load the data from the database, you need to enable the read-through mode and set the cacheStoreFactory property of CacheConfiguration. You can set these values either in your Spring XML configuration file or programmatically.

<bean class="org.springframework.jdbc.datasource.DriverManagerDataSource" id="dataSource">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"></property>
    <property name="url" value="jdbc:mysql://localhost:3306/mydbname"></property>
    <property name="username" value="username"></property>
    <property name="password" value="passwd"></property>
</bean>

<bean class="org.apache.ignite.configuration.IgniteConfiguration" id="ignite.cfg">
    <property name="cacheConfiguration">
        <list>
            <bean class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="name" value="personCache"></property>
                <!-- Enable readThrough-->
                <property name="readThrough" value="true"></property>
                <property name="writeThrough" value="true"></property>
                <!-- Set cacheStoreFactory-->
                <property name="cacheStoreFactory">
                    <bean class="javax.cache.configuration.FactoryBuilder" factory-method="factoryOf">
                        <constructor-arg value="myexamples.store.PersonStore"></constructor-arg>
                    </bean>
                </property>
                <property name="queryEntities">
                    <list>
                        <bean class="org.apache.ignite.cache.QueryEntity">
                            <property name="keyType" value="java.lang.Long"></property>
                            <property name="valueType" value="ignite.myexamples.model.Person"></property>
                            <property name="fields">
                                <map>
                                    <entry key="id" value="java.lang.Long"></entry>
                                    <entry key="name" value="java.lang.String"></entry>
                                    <entry key="orgId" value="java.lang.Long"></entry>
                                    <entry key="salary" value="java.lang.Integer"></entry>
                                </map>
                            </property>
                        </bean>
                    </list>
                </property>
            </bean>
        </list>
    </property>

    <property name="peerClassLoadingEnabled" value="true"></property>

    <!-- Other Ignite configurations-->
    ...

</bean>

5. Implement CacheStore

Now that we have our model, maven dependencies, and cache configurations in place, it’s time to implement the store. To load the data from the database, loadCache() and load() methods of the CacheStore interface should be implemented. 

public class PersonStore implements CacheStore<Long, Person> {

    @SpringResource(resourceName = "dataSource")
    private DataSource dataSource;

    // This method is called whenever IgniteCache.loadCache() method is called.
    @Override
    public void loadCache(IgniteBiInClosure<Long, Person> clo, @Nullable Object... objects) throws CacheLoaderException {
        System.out.println(">> Loading cache from store...");

        try (Connection conn = dataSource.getConnection()) {
            try (PreparedStatement st = conn.prepareStatement("select * from PERSON")) {
                try (ResultSet rs = st.executeQuery()) {
                    while (rs.next()) {
                        Person person = new Person(rs.getLong(1), rs.getLong(2), rs.getString(3), rs.getInt(4));

                        clo.apply(person.getId(), person);
                    }
                }
            }
        }
        catch (SQLException e) {
            throw new CacheLoaderException("Failed to load values from cache store.", e);
        }
    }

    // This method is called whenever IgniteCache.get() method is called.
    @Override
    public Person load(Long key) throws CacheLoaderException {
        System.out.println(">> Loading person from store...");

        try (Connection conn = dataSource.getConnection()) {
            try (PreparedStatement st = conn.prepareStatement("select * from PERSON where id = ?")) {
                st.setString(1, key.toString());

                ResultSet rs = st.executeQuery();

                return rs.next() ? new Person(rs.getLong(1), rs.getLong(2), rs.getString(3), rs.getInt(4)) : null;
            }
        }
        catch (SQLException e) {
            throw new CacheLoaderException("Failed to load values from cache store.", e);
        }
    }

    // Other CacheStore method implementations.
    …
}

For convenience purposes, Ignite also provides it users with CacheStoreAdapter class that has a default implementation for some of the CacheStore methods - loadAll()writeAll(), and deleteAll().

6. Load Cache

Here is a sample PersonStoreExample.java class that makes a call to IgniteCache.loadCache()  method which internally delegates the call to CacheStore.loadCache() method (which we implemented in the previous step).

public class PersonStoreExample {
    public static void main(String[] args) throws IgniteException {
        Ignition.setClientMode(true);

        try (Ignite ignite = Ignition.start("config/cluster-config.xml")) {
            try (IgniteCache<Long, Person> cache = ignite.getOrCreateCache("personCache")) {
                // Load cache with data from the database.
                cache.loadCache(null);

                // Execute query on cache.
                QueryCursor<List<?>> cursor = cache.query(new SqlFieldsQuery(
                        "select id, name from Person"));

                System.out.println(cursor.getAll());
            }
        }
    }
}

7. Start Ignite Cluster

From the command shell, lead yourself to the Ignite installation folder and start the server nodes, using the following command:

$ bin/ignite.sh <path-to-Spring-XML-configuration-file>

Make sure that PersonStore.java is in the class path of Ignite. To do so, you can set the USER_LIBS environment variable, or drop the project jar into the libs folder of your Ignite installation. 

8. Output

From your IDE, run PersonStoreExample.java.

Image title


For more information, documentation, and screencasts visit the Apache Ignite website.

Interested in reducing database costs by moving from Oracle Enterprise to open source subscription?  Read the total cost of ownership (TCO) analysis. Brought to you in partnership with MariaDB.

Topics:
caching ,apache ignite ,tutorial ,database ,data grid ,compute grid ,sql

Published at DZone with permission of Dmitriy Setrakyan, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}