DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Data
  4. Moving Streaming Analytics Out of the Data Center

Moving Streaming Analytics Out of the Data Center

Why moving analytics out of the traditional data center benefits cybersecurity, fraud, maintenance, and customer experience.

Mark Lochbihler user avatar by
Mark Lochbihler
·
May. 29, 16 · Opinion
Like (1)
Save
Tweet
Share
4.59K Views

Join the DZone community and get the full member experience.

Join For Free

this blog focuses on moving streaming analytics outside the confines of the traditional data center. moving streaming analytics closer to where data originates can be accomplished by leveraging an enterprise-grade data movement application, married with an extremely lightweight streaming engine. this combination is being used by forward-looking organizations to solve usage cases in a number of areas, including:

  • cyber security – identify a malicious intrusion before or as it occurs.
  • fraud – analyze streaming transactions to determine which needs immediate attention.
  • predictive maintenance – predict outlier conditions from streaming machine and sensor data.
  • customer experience and marketing – use streaming data insights to personalize interactions.
  • stream data management – transform and clean data in motion, storing only what you need.

ioat (internet of anything) data includes any new data source generated from sensors and machines, server logs, clickstream web application servers, social media, as well as files and email. by reacting rapidly, a high degree of analytic intelligence such as ioat data can be collected from its origin to help companies gain fast insights to outpace their competition. whether it’s personalizing a “next best offer” or providing an immediate treatment to an early warning alert, ioat data is only beginning to impact bottom line revenues. the time to act is now, but what components do we need to add to our modern data applications?

hortonworks data flow

hortonworks data flow (hdf), powered by apache nifi, is a data application platform designed to solve data acquisition and delivery challenges, inside and outside the data center. it provides a fast, easy and secure way to move data from anywhere it originates to anywhere it needs to go. hdf has a simple gui command and control for building “data workflows” so there is no need to write custom data movement scripts. hdf provides simple event processing (sep) out of the box to curate data payloads as they move from one place to another.

image title

figure 1 : act fast on streaming data with hdf and sas esp

sas event stream processing

sas event stream processing (esp) analyzes and understands streaming data as it is being generated, detects patterns of interest as they occur, and provides the necessary instructions to take the correct actions; i.e. what alerts to issue and what portions of data should be retained for further investigation.

hdf and sas esp

by incorporating sas esp into our edge originating hdf based data workflows, we are able to inject complex event processing (cep) into any ioat application. it’s important to note that both hdf and sas esp are ideal candidates to run on edge node gateways, which typically have very small memory footprints and run lightweight linux kernels. since hdf and sas esp are architected to run from the smallest of devices large clustered configurations, they are a perfect compliment both inside and outside the data center.

summary

in summary, hdf, powered by apache nifi, is a secure, reliable enterprise data movement platform, which provides simple event processing (sep) processors out of the box. hdf is architected to run from the data center to the edge of an ioat framework and back. sas esp is an extremely fast streaming analytics engine, also architected to run in the data center and out to an edge node device and back. sas esp and hdf are seamlessly integrated via sas esp nifi processors. together, they can provide immediate, streaming, deep actionable intelligence for improved customer experience and lower cost operations.

Data (computing) Analytics hadoop Stream processing Web application Event stream processing

Published at DZone with permission of Mark Lochbihler, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • How to Create a Real-Time Scalable Streaming App Using Apache NiFi, Apache Pulsar, and Apache Flink SQL
  • Tech Layoffs [Comic]
  • The 31 Flavors of Data Lineage and Why Vanilla Doesn’t Cut It
  • Deploying Java Serverless Functions as AWS Lambda

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: