Real-Time Streaming Analytics With Oracle GoldenGate

Big Data continues to evolve with an increasing number of solutions running in-memory with real-time data processing. The combination of real-time data and processing, and real-time actions defines the new category of Streaming Analytics. Streaming Analytics responds to customer (or devices) in real time, with offers on a website, a SMS message, or even an outbound call from a customer support center. 

Real-Time Streaming Analytics Architecture

Steaming Analytics begins with a combination of events that define a scenario, which triggers a real-time action. Scenarios include real-time activities, and can include historical data, and include criteria such as a time window:

• A wireless customer experiences four dropped calls within a 12-hour window.

• A wireless customer experiences four dropped calls within a 12-hour window, calls into customer support but leaves the queue prior to being helped

• A customer is approaching expiration of a service contract and is currently on the web page for contract renewal

Once scenarios are defined, data sources are identified and “listeners” are integrated that identify and parse events, and deliver formatted messages to the event processing engine. The processing engine contains the business rules for each scenario, and provides scenario recognition in milliseconds, triggering the corresponding action. 

The Architecture for Real-Time Streaming Analytics: 

Turning Real-Time Data Into Action

Real-time event strategies can include virtually any data source, such as network events, system reboots, calls, web session activity, geolocation, and can be enhanced with historical data from CRM and other systems. Where Big Data and Analytics focus on capturing data and historical analysis, Streaming Analytics focuses on real-time event recognition and action.  

The Streaming Analytics engine processes events in real time, combining events with historical data.  Scenarios are recognized in less than 50 milliseconds; actions are triggered and recorded in persistent storage for dashboards and visual display.  

Oracle GoldenGate as a Real Time Data Source

GoldenGate is recognized as a leading solution for real-time database replication and integration. GoldenGate separates extraction and replication processes, with records of database changes (inserts, updates, deletes) in log files (“trail” files). GoldenGate trail files are rich source of real time events and can be used without impacting database operations.    

GoldenGate trail files are written either locally or remotely, depending on the GoldenGate configuration.  (“extract trail” or “remote trail,” respectively). EVAM utilizes a “GG_ExtractListener” to process ASCII trail files continuously, similar to a Linux tail command. The listener parses, formats, and feeds events to the Streaming Analytics engine.  

The “GG_ExtractListener” is an example of a lightweight configurable java application that can be run on any JVM. The GG_ExtractListener requires only read permissions on the trail files, and can support use on the source system ExtractTrails (1), or on a remote machine RemoteTrails (2).  

Conclusions

Smart organizations are taking the next step beyond historical big data analysis to drive real-time customer engagement with streaming analytics. Any organization using GoldenGate can easily tap into the real-time events to define events and scenarios, with corresponding actions.