Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Closed Loop: Machine Learning and Real Time Stream Processing

DZone's Guide to

Closed Loop: Machine Learning and Real Time Stream Processing

A summary of Kai Wähner's talk at Voxxed Zurich about machine learning and real time stream processing with Apache Spark, TIBCO, Apache Storm, and other tools.

· Big Data Zone ·
Free Resource

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

In March 2016, I had a talk at Voxxed Zurich about “How to Apply Machine Learning and Big Data Analytics to Real Time Processing”.

Image title

Finding Insights With R, H20, Apache Spark MLlib, PMML, and TIBCO Spotfire

Big Data” is currently a big hype. Large amounts of historical data are stored in Hadoop or other platforms. Business Intelligence tools and statistical computing are used to draw new knowledge and to find patterns from this data, for example for promotions, cross-selling or fraud detection. The key challenge is how these findings can be integrated from historical data into new transactions in real time to make customers happy, increase revenue or prevent fraud.

Putting Analytic Models Into Action With Apache Storm, Flink, Spark Streaming, or TIBCO StreamBase

Fast Data” via stream processing is the solution to embed patterns – which were obtained from analyzing historical data – into future transactions in real-time. The following slide deck uses several real world success stories to explain the concepts behind stream processing and its relation to Apache Hadoop and other big data platforms. I discuss how patterns and statistical models of R, Apache Spark MLlib, H20, and other technologies can be integrated into real-time processing using open source stream processing frameworks (such as Apache Storm, Spark Streaming or Flink) or products (such as IBM InfoSphere Streams or TIBCO StreamBase). A great benefit is if a framework or tool already offers out-of-the-box connectors to execute analytic models without need for rebuilding the model respecively rewriting any code. 

Slide Deck and Video Recording From Voxxed Zurich 2016

Here is the slide deck:

How to Apply Machine Learning with R, H20, Apache Spark MLlib or PMML to Real Time Streaming Analytics from Kai Wähner

The video recording of this talk is available on Youtube:

A live demo showed the complete development lifecycle combining analytics with TIBCO Spotfire, machine learning via R and stream processing via TIBCO StreamBase and TIBCO Live Datamart.

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.

Topics:
big data ,stream processing ,r language ,spark ,tibco ,open source ,storm ,machine learning ,analytics ,fast data

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}