Over a million developers have joined DZone.

Continuous Queries

· Big Data Zone

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks

I haven't blogged in a while mainly because of fundraising, lots of hiring, and launching new products (In-Memory StreamingIn-Memory Hadoop Accelerator, and In-Memory NoSQL Accelerator) that kept us busy at GridGain in the past several months. Now that most of it is behind us, I will try to write a blog a week.

In this post I will talk about continuous querying capabilities at GridGain. GridGain already supports SQL, TEXT, and Full Scan queries, but what if you need to continuously get query results as the data gets updated or changed in In-Memory Database (IMDB)? Continuous queries provide the mechanism to do it.

Note that continuous queries are fully based on callback notifications. This way we avoid any time window between iterating through existing result set and getting callbacks on future updates. Instead, you get callbacks for all the data residing in IMDB, whether it's current or future data.

Let's take a look at the example:

// Create continuous query.
GridCacheContinuousQuery<String, UserData> qry = cache.createContinuousQuery();
// Specify the callback to get query results. Note that results will be coming
// until query is cancelled.
qry.callback(new GridPredicate2<UUID, Collection<Map.Entry<String, UserData>>>() {
    @Override public boolean apply(UUID nodeId, Collection<Map.Entry<String, UserData>> entries) {
        for (Map.Entry<String, UserData> e : entries)
        return true;
// Specify the predicate to filter results on all participating nodes.
qry.filter(new GridPredicate2<String, UserData>() {
    @Override public boolean apply(String userId, UserData userData) {
        return userData.getLastUpdateTime().getTime() >= cal.getTimeInMillis();
// Run the query. We'll get current results and
// further updates in a callback.

On line 2 we create a new continuous query. Lines 6-13 handle the callback, in which we'll get current query results and further updates as cache data changes. Lines 16-20 are just our original SQL query converted into Java predicate. The query starts on line 24.

Hortonworks Sandbox is a personal, portable Apache Hadoop® environment that comes with dozens of interactive Hadoop and it's ecosystem tutorials and the most exciting developments from the latest HDP distribution, brought to you in partnership with Hortonworks.


Published at DZone with permission of Dmitriy Setrakyan, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}