Event-Driven Architecture 101: Continuous Query Cache

DZone 's Guide to

Event-Driven Architecture 101: Continuous Query Cache

Learn all about Oracle CQC in this look at event driven architecture. Check out what event driven architecture is, examples of events, and EDA characteristics.

· Integration Zone ·
Free Resource

What is an Event Driven Architecture?

Events are everywhere and trigger a change of state. Processing of events (business or system) in a real-time scenario requires both analysis and acting on that event (reactive and proactive responses)

Events are sometimes predictable, but often unpredictable in nature. That’s why the proactive and
reactive ways of handling events are imminent.
  A. Reactive triggering – Knows what needs to be done once the event is received
  B. Proactive triggering – Needs the capabilities of pattern matching, intelligent routing, and decision making to respond quickly

Examples of Events

Alarm rings (Reactive triggering)
Create Account (Reactive triggering)
Send warning signal before storm comes (Proactive triggering)

What’s Next?

Need a system/architecture which is comprehensive, enterprise-wise solution for capture, process, and monitor the events and build the capability to predict changes in advance and take intelligent decisions.
Here we go……
Event Driven Architecture (EDA) is key to both proactive and reactive mode of triggering point based on event-based integration.

Events are key to EDA.

EDA Characteristics

Sense, Analyze, Response - A cyclic procedure with the capability to self-learn and use intelligence.

  • We detect events across distributed environments in real-time
  • Then analyze that event, aggregate events from multiple sources, and compare reality with expectations
  • We update expectations and then build a pattern oriented recognition to go in the same circle

EDA works through a network called Event Processing Networks. The following is the typical flow for any event processor architecture.

Event Source -> Event Channels -> Event Processor -> Event Sinks (Target System)

Image title

Image courtesy of Oracle

Type of Events

Events are loosely coupled in nature

  •     Don’t tie an action to the consequences

Distributed in nature

  •     Both event sources and targets can be distributed over the network

Reactive to any event

  •     Anything notable happens and trigger an action
  •     Events can have nested link – one event can trigger another one and so on

Events are push based rather than pull based

  •     As soon as event comes, it is being processed without waiting
  •     Source doesn’t determine the operation, the recipient makes the choice

Basically, there are two types of events in EDA:

  • Technical (Transfer entity state from source to target system)
  • Business (Reaction to the behavior exhibited by the event)

Business events can be triggered on any update to entity state or understanding patterns which may lead to detect any unpredictable situation – support simple and complex event processing

Classification of Events

There are three classifications for EDA.

Simple Event processing – Any particularly notable thing happens and processed downstream

Stream Event Processing – All events are delivered to the system and screened for processing (streams of data processed with continuous inputs). Multiple ordered events in one or multiple streams.

Complex Event Processing – Confluence of events for making correlations (casual, temporal, and spatial). The characteristics include matching of patterns, filtering, and predictive analysis, hierarchical events, aggregation of events etc. The events may be unrelated.

Overview of Coherence

An in-memory enterprise highly-scalable and low-latency data and computational grid which can support processing of data in a distributed clustered environment to reduce the cost of accessing data.

Main Usage

  • Off-loading expensive and difficult-to-scale backend systems like database and mainframe
  • exTreme Transaction Processing through partitioned/distributed processing of data to get very high throughput
  • A high processing computational grid which uses the intelligent data processing through in-memory data architecture
  • Backed by tangosol based pof (portable object format) structure which facilitates an efficient and high-performing data-serialization process
  • Great data integration platform – facilitates to integrate and load data from different data sources like database, mainframe, ERP, CRM systems to provide a uniform data access layer for the client
  • A foundational support for Event Driven Architecture and can be integrated quickly with any messaging, ESB, Complex Event Processing

Coherence in EDA Space

How can Coherence support the EDA features?

  • Can act as the event source
  • Can hold the entity state (Coherence objects)
  • Can hold the event channels (streams)
  • Can act as the event processor
  • Can support the Event listeners (Core to Continuous Query Cache) – Act as the event sink/targets

Coherence has the following building blocks for EDA.

  • MapListerner – Client side “post” events
  • LiveObjects
  • EventInterceptor

There are other event-based mechanisms like “Backing Map Listener” or server side “pre-events” through Triggers

High Level on EventInterceptor

EventInterceptor are used on live events

  • Live events are based on an event-based model where events represent the observable occurrences of cluster operations
  • Live events can be a partitioned cache event, partitioned cache service, or application events

EventInterceptor can filter events based on type (uses Java generics)

Handles cases for pre-commit and post-commit cases

Registration can happen through configuring the cache file or programmatically

The filter is created like the following

EntryEvent of type INSERTING (pre-commit) or INSERTED (post-commit)

The EventInterceptor provides the onEvent() method to consume the event and take necessary actions

The live events have been introduced in Oracle Coherence 12c version

What is Continuous Query Cache?

The continuous query cache combines a query result with a continuous stream of related events to maintain an up-to-date query result in a real-time.

It is an ideal building block for Complex Event Processing (CEP) systems and event correlation engine.

Continuous Query Cache (CQC) is similar to a “DB Materialized View” which has physical existence under the logical view. Data is backed by Coherence Near Cache through filters to pass only relevant information in the CQC region. The continuous flow of data from CQC can be tapped and process through inbuilt trigger based methods.

Coherence creates synchronous/asynchronous listeners on the CQC based on the filters. When any message comes to Near Cache, it gets intercepted by listeners to apply the filters. Once the filter is passed, the data is moved to CQC region for processing of selected in-memory data.

The continuous query cache actually maintains its data in an up-to-date manner.

Continuous Query Cache Flow

Event sources can be anything which puts data in Near Cache

Filters are created to specify which kind of data to flow continuously in CQC

Clients access data in CQC through listeners

CQC provides the implementation of “MapListener” for triggering event-driven flow

Trigger methods are CRUD (insert, update, and delete) in nature

Image title

CQC Features

  • Conceptually similar to Near Cache (A front cache and back cache) in Coherence
  • Can be Read-only or read-write
  • Can be based on Synchronous or Asynchronous Listeners
  • A zero-latency front cache which is always up-to-date and slower but bigger back cache
  • Keeps a subset of data through filters (Therefore, the data in CQC is mostly lesser than Near Cache)
  • Created through Coherence API only (no configuration based initialization)
  • CQC implements Observable which means clients can register and listen to the change in state for the CQC

Implementation of CQC

Implements from Coherence Named Cache. Therefore, by default inherits many useful features

  • Work as just a Map (like normal Java HashMap)
  • Event based since implementing ObservableMap
  • Identify-based cluster-wide locking due to implementing ConcurrentMap
  • Querying Map due to implementing QueryMap
  • Distributed Parallel Processing and Aggregating due to implementing InvocableMap
  • CQC needs the ContinuousQueryCache class to be instantiated with the Coherence NamedCache, filter, and listener
  • The filter is defined criteria based on which data moving to Coherence will be available in CQC for further processing
  • The listener implements CQC MapListener interface (the source of event driven architecture in Coherence).
  • The listener provides three methods to operate on insert/update/delete operations.

Sample Code:

1. new ContinuousQueryCache(CACHE, queryFilter, cqcListener);

2. final Filter eventFinishedFilter = QueryHelper.createFilter("isEventFinished()");

3. public void entryInserted(final MapEvent event) {}

Performance Challenges and Solution

There following are some general guidelines when using CQC to process data.

  • Create the CQC object with the cqcListener instead of using addListener method.in order to avoid missing any events between creating the ContinuousQueryCache object and adding the listener to that
  • Only store keys instead of the whole object with key and value
  • By default listener in CQC delivers events in asynchronous mode
coherence, event driven architecture, integration

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}