Over a million developers have joined DZone.

Apache Spark Complex Event Processing, Training and SparkSQL Datawarehouse

A link sheet for those who want to learn more about using Apache Spark and data warehousing.

· Big Data Zone

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks

Apache Spark and Machine learning can solve a lot of problems and can discover patterns leading to rules replacement.  But for some simple cases and for legacy rules requires, you may still need to hand develop rules and implement in Java.  For those cases, Drools can be used with Spark.   Besides machine learning, CEP or complex event processing can be used as a replacement for standard IF-THEN-ELSE style hardcoded rules.

Siddhi Complex Event Processing Engine

Use Complex Event Processing (CEP) instead of rules:

Decision CEP Engine - complex event processing platform build on spark streaming

Architecture Guide

Spark plus Drools


Break up your kafka messages

Rules / File Matching


Business Rules - JRules Data Mining

There's a ton of great presentations, webinars and short online courses for learning Spark.   Here are a few that I recommend.

More Free Spark Training

Apache Spark Essentials

Build and Monitor Spark Applications

Create Data Pipelines with Spark

MapR Free Spark Training

When implementing a SQL Datawarehouse on Hadoop with Spark, here are a few useful starters.

SQL Data Warehouse With Spark

SparkSQL Datawarehouse

Building a Datawarehouse

Spark Streaming for Robust Apps

Spark Summit Keynote

Memory-centric Distributed Storage System

Alluxio (Tachyon) - in memory

New Fast SQL Project

Alluxio is proving to be a great general purpose in memory file system.

Hortonworks Sandbox is a personal, portable Apache Hadoop® environment that comes with dozens of interactive Hadoop and it's ecosystem tutorials and the most exciting developments from the latest HDP distribution, brought to you in partnership with Hortonworks.

big data,spark,apache spark,spark sql

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}