DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Databases
  4. Apache Ignite: How to Read Data From Persistent Store

Apache Ignite: How to Read Data From Persistent Store

In this post, author Dmitriy Setrakyan shows you how to load data from a MySQL database into an Ignite distributed cache. Read on and see how it's done.

Dmitriy Setrakyan user avatar by
Dmitriy Setrakyan
·
Jun. 03, 16 · Tutorial
Like (13)
Save
Tweet
Share
35.48K Views

Join the DZone community and get the full member experience.

Join For Free

Data can be loaded directly from any persistent store into Apache Ignite caches. This example shows you how to load data from a MySQL database into an Ignite distributed cache. Here, I am assuming that you already have Apache Ignite installed on your system. If not, you can go through this tutorial first.

1. Sample PERSON Table

To start with, here is what the PERSON data in my database looks like: 

Image title

2. Model 

Here is a sample Person.java class corresponding to the PERSON table in the database.

public class Person {

    private long id;

    private long orgId;

    private String name;

    private int salary;

    // Constructor
    …
    // Getter and Setter methods
    …
}

3. Maven Dependency

I have specified the following dependencies in my project’s pom.xml: 

<dependency>
    <groupid>org.apache.ignite</groupid>
    <artifactid>ignite-core</artifactid>
    <version>1.5.0.final</version>
</dependency>

<dependency>
    <groupid>org.apache.ignite</groupid>
    <artifactid>ignite-spring</artifactid>
    <version>1.5.0.final</version>
</dependency>

<dependency>
    <groupid>mysql</groupid>
    <artifactid>mysql-connector-java</artifactid>
    <version>5.1.6</version>
</dependency>

4. Configure Read-Through

To load the data from the database, you need to enable the read-through mode and set the cacheStoreFactory property of CacheConfiguration. You can set these values either in your Spring XML configuration file or programmatically.

<bean class="org.springframework.jdbc.datasource.DriverManagerDataSource" id="dataSource">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"></property>
    <property name="url" value="jdbc:mysql://localhost:3306/mydbname"></property>
    <property name="username" value="username"></property>
    <property name="password" value="passwd"></property>
</bean>

<bean class="org.apache.ignite.configuration.IgniteConfiguration" id="ignite.cfg">
    <property name="cacheConfiguration">
        <list>
            <bean class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="name" value="personCache"></property>
                <!-- Enable readThrough-->
                <property name="readThrough" value="true"></property>
                <property name="writeThrough" value="true"></property>
                <!-- Set cacheStoreFactory-->
                <property name="cacheStoreFactory">
                    <bean class="javax.cache.configuration.FactoryBuilder" factory-method="factoryOf">
                        <constructor-arg value="myexamples.store.PersonStore"></constructor-arg>
                    </bean>
                </property>
                <property name="queryEntities">
                    <list>
                        <bean class="org.apache.ignite.cache.QueryEntity">
                            <property name="keyType" value="java.lang.Long"></property>
                            <property name="valueType" value="ignite.myexamples.model.Person"></property>
                            <property name="fields">
                                <map>
                                    <entry key="id" value="java.lang.Long"></entry>
                                    <entry key="name" value="java.lang.String"></entry>
                                    <entry key="orgId" value="java.lang.Long"></entry>
                                    <entry key="salary" value="java.lang.Integer"></entry>
                                </map>
                            </property>
                        </bean>
                    </list>
                </property>
            </bean>
        </list>
    </property>

    <property name="peerClassLoadingEnabled" value="true"></property>

    <!-- Other Ignite configurations-->
    ...

</bean>

5. Implement CacheStore

Now that we have our model, maven dependencies, and cache configurations in place, it’s time to implement the store. To load the data from the database, loadCache() and load() methods of the CacheStore interface should be implemented. 

public class PersonStore implements CacheStore<Long, Person> {

    @SpringResource(resourceName = "dataSource")
    private DataSource dataSource;

    // This method is called whenever IgniteCache.loadCache() method is called.
    @Override
    public void loadCache(IgniteBiInClosure<Long, Person> clo, @Nullable Object... objects) throws CacheLoaderException {
        System.out.println(">> Loading cache from store...");

        try (Connection conn = dataSource.getConnection()) {
            try (PreparedStatement st = conn.prepareStatement("select * from PERSON")) {
                try (ResultSet rs = st.executeQuery()) {
                    while (rs.next()) {
                        Person person = new Person(rs.getLong(1), rs.getLong(2), rs.getString(3), rs.getInt(4));

                        clo.apply(person.getId(), person);
                    }
                }
            }
        }
        catch (SQLException e) {
            throw new CacheLoaderException("Failed to load values from cache store.", e);
        }
    }

    // This method is called whenever IgniteCache.get() method is called.
    @Override
    public Person load(Long key) throws CacheLoaderException {
        System.out.println(">> Loading person from store...");

        try (Connection conn = dataSource.getConnection()) {
            try (PreparedStatement st = conn.prepareStatement("select * from PERSON where id = ?")) {
                st.setString(1, key.toString());

                ResultSet rs = st.executeQuery();

                return rs.next() ? new Person(rs.getLong(1), rs.getLong(2), rs.getString(3), rs.getInt(4)) : null;
            }
        }
        catch (SQLException e) {
            throw new CacheLoaderException("Failed to load values from cache store.", e);
        }
    }

    // Other CacheStore method implementations.
    …
}

For convenience purposes, Ignite also provides it users with CacheStoreAdapter class that has a default implementation for some of the CacheStore methods - loadAll(), writeAll(), and deleteAll().

6. Load Cache

Here is a sample PersonStoreExample.java class that makes a call to IgniteCache.loadCache()  method which internally delegates the call to CacheStore.loadCache() method (which we implemented in the previous step).

public class PersonStoreExample {
    public static void main(String[] args) throws IgniteException {
        Ignition.setClientMode(true);

        try (Ignite ignite = Ignition.start("config/cluster-config.xml")) {
            try (IgniteCache<Long, Person> cache = ignite.getOrCreateCache("personCache")) {
                // Load cache with data from the database.
                cache.loadCache(null);

                // Execute query on cache.
                QueryCursor<List<?>> cursor = cache.query(new SqlFieldsQuery(
                        "select id, name from Person"));

                System.out.println(cursor.getAll());
            }
        }
    }
}

7. Start Ignite Cluster

From the command shell, lead yourself to the Ignite installation folder and start the server nodes, using the following command:

$ bin/ignite.sh <path-to-Spring-XML-configuration-file>

Make sure that PersonStore.java is in the class path of Ignite. To do so, you can set the USER_LIBS environment variable, or drop the project jar into the libs folder of your Ignite installation. 

8. Output

From your IDE, run PersonStoreExample.java.

Image title


For more information, documentation, and screencasts visit the Apache Ignite website.

Apache Ignite Data (computing) Database

Published at DZone with permission of Dmitriy Setrakyan, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Fraud Detection With Apache Kafka, KSQL, and Apache Flink
  • Beginners’ Guide to Run a Linux Server Securely
  • A Beginner's Guide to Back-End Development
  • Exploring the Benefits of Cloud Computing: From IaaS, PaaS, SaaS to Google Cloud, AWS, and Microsoft

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: