DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Data
  4. Real-Time Streaming Analytics With Oracle GoldenGate

Real-Time Streaming Analytics With Oracle GoldenGate

The name of the game is speed. Take a look at how you can use Oracle's GoldenGate as part of a real-time analytics architecture for quick, efficient data turnaround.

Paul Stanton user avatar by
Paul Stanton
·
Oct. 18, 16 · Opinion
Like (5)
Save
Tweet
Share
6.18K Views

Join the DZone community and get the full member experience.

Join For Free

Big Data continues to evolve with an increasing number of solutions running in-memory with real-time data processing. The combination of real-time data and processing, and real-time actions defines the new category of Streaming Analytics. Streaming Analytics responds to customer (or devices) in real time, with offers on a website, a SMS message, or even an outbound call from a customer support center. 

Real-Time Streaming Analytics Architecture

Steaming Analytics begins with a combination of events that define a scenario, which triggers a real-time action. Scenarios include real-time activities, and can include historical data, and include criteria such as a time window:

  • A wireless customer experiences four dropped calls within a 12-hour window.

  • A wireless customer experiences four dropped calls within a 12-hour window, calls into customer support but leaves the queue prior to being helped

  • A customer is approaching expiration of a service contract and is currently on the web page for contract renewal

Once scenarios are defined, data sources are identified and “listeners” are integrated that identify and parse events, and deliver formatted messages to the event processing engine. The processing engine contains the business rules for each scenario, and provides scenario recognition in milliseconds, triggering the corresponding action. 

The Architecture for Real-Time Streaming Analytics: 

Image title

Turning Real-Time Data Into Action

Real-time event strategies can include virtually any data source, such as network events, system reboots, calls, web session activity, geolocation, and can be enhanced with historical data from CRM and other systems. Where Big Data and Analytics focus on capturing data and historical analysis, Streaming Analytics focuses on real-time event recognition and action.  

The Streaming Analytics engine processes events in real time, combining events with historical data.  Scenarios are recognized in less than 50 milliseconds; actions are triggered and recorded in persistent storage for dashboards and visual display.  

Oracle GoldenGate as a Real Time Data Source

GoldenGate is recognized as a leading solution for real-time database replication and integration. GoldenGate separates extraction and replication processes, with records of database changes (inserts, updates, deletes) in log files (“trail” files). GoldenGate trail files are rich source of real time events and can be used without impacting database operations.    

GoldenGate trail files are written either locally or remotely, depending on the GoldenGate configuration.  (“extract trail” or “remote trail,” respectively). EVAM utilizes a “GG_ExtractListener” to process ASCII trail files continuously, similar to a Linux tail command. The listener parses, formats, and feeds events to the Streaming Analytics engine.  

The “GG_ExtractListener” is an example of a lightweight configurable java application that can be run on any JVM. The GG_ExtractListener requires only read permissions on the trail files, and can support use on the source system ExtractTrails (1), or on a remote machine RemoteTrails (2).  

Image title

Conclusions

Smart organizations are taking the next step beyond historical big data analysis to drive real-time customer engagement with streaming analytics. Any organization using GoldenGate can easily tap into the real-time events to define events and scenarios, with corresponding actions.  

Analytics Big data Event Database

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • How To Check Docker Images for Vulnerabilities
  • A Brief Overview of the Spring Cloud Framework
  • How Observability Is Redefining Developer Roles
  • Fraud Detection With Apache Kafka, KSQL, and Apache Flink

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: