Apache Spark Complex Event Processing, Training and SparkSQL Datawarehouse

Apache Spark and Machine learning can solve a lot of problems and can discover patterns leading to rules replacement.  But for some simple cases and for legacy rules requires, you may still need to hand develop rules and implement in Java.  For those cases, Drools can be used with Spark.   Besides machine learning, CEP or complex event processing can be used as a replacement for standard IF-THEN-ELSE style hardcoded rules.

Siddhi Complex Event Processing Engine

Use Complex Event Processing (CEP) instead of rules:

Decision CEP Engine - complex event processing platform build on spark streaming

Architecture Guide

Spark plus Drools

Sprue

Break up your kafka messages

Rules / File Matching

Herzberg

Business Rules - JRules Data Mining

There's a ton of great presentations, webinars and short online courses for learning Spark.   Here are a few that I recommend.

More Free Spark Training

Apache Spark Essentials

Build and Monitor Spark Applications

Create Data Pipelines with Spark

MapR Free Spark Training

When implementing a SQL Datawarehouse on Hadoop with Spark, here are a few useful starters.

SQL Data Warehouse With Spark

SparkSQL Datawarehouse

Building a Datawarehouse

Spark Streaming for Robust Apps

Spark Summit Keynote

Memory-centric Distributed Storage System

Alluxio (Tachyon) - in memory

New Fast SQL Project

Alluxio is proving to be a great general purpose in memory file system.