FlinkForward Conference 2016
My friend Julian Hyde does a very interesting talk on
StreamingSQL which is going to be huge in many of the streaming systems. With Apache Calcite, the hope is that all the different Streaming SQLs will be using the same tools, syntax and libraries so we can have compatibility across streaming libraries.
Another sign that Apache Flink is becoming more mature,
Apache Flink Security Enhancements.
One of the interesting features of Flink is
Code Generation in Serializers and Comparators.
Flink is now able to work with Apache Zeppelin, making on par with the ease of data exploration that you get with Apache Spark.
Spark Summit EU 2016
Spark for ETL (Nov 2016)
Spark and Lucene for near-realtime predictive model building
Problem Solving Recipes Learned from Supporting Spark
Elastic Streaming (Spark)
Sparkling Insights at Elsevier
Spark - Universal Computation Engine for Processing Oil Industry Data
Spark and SAP HANA Vora
Online Learning With Structured Streaming VIDEO
Adopting Dataframes and Parquet in an already existing datawarehouse
Spark ML with Redis
Sparksheet - Transforming Spreadsheets into Spark Data Frames ( GitHub)
Boosting Spark Performance on Many-Core Machines
Performance Characterization of In-Memory Data Analytics on a Scale-up Server
Extending Spark with Java Agents ( GitHub)
SparkSQL 2.0/2.1 Experiences using TPC-DS
Apache Spark and Object Stores - What you need to know
Deep Dive into Catalyst: Apache Spark's Optimizer
Apache Ignite with Apache Spark
Sparkling Water 2.0: H2O Deep Water for working with TensorFlow, MXNet, and Caffe and with Zeppelin. This really expands what you can do on the JVM.
Effective Spark with Alluxio
Mastering Spark Unit Testing
Online Structured Streaming
Apache Spark 2.0 Performance Improvements Investigated with Flame Graphs ( Details)
Streaming Outlier Analysis for Fun and Scalability
IoT With NIFI
For complex IoT programming, you need a combination of Apache NiFi, Storm, and Kafka Together. This is a great talk on Edge Intelligence IoT with MiniFi. Installing MiniFI on Your Device, will allow you to access all kinds of IoT devices and stream their data into your corporate NIFI servers. MiniFi in C++ - Very early, but fast small C++ compiled code for devices without Java.
Hadoop Summit Tokyo Oct 2016
Keynote on Connectivity, Big Data, and IoT
VIDEO Apache NiFi 1.0 in a Nutshell
Zeppelin with Matlib Plots : A cool way to show more complex graph plots.
Hadoop Summit Tokyo Notes
Spark Tuning for Enterprise System Administrators (
Cheat Sheet) for Spark 1.6
Apache Spark Crash Course : this is a great introduction to learning Apache Spark.
Apache HiveMall on Spark Dataframes for doing Machine Learning in your Hadoop Database
With expanded talk on Apache HiveMall: ML Lib for Hive/Spark/Pig
Apache NiFi Crash Course: a great introduction to learning Apache NiFi.
Streamline Hadoop DevOps with Apache Ambari
Hadoop Sandbox Workshop
Other Good Recent Talks
Practical Kerberos with Apache HBase
A nice overview and dive into Hadoop, Hive, HBase and NoSQL
Credit Card Fraud Prevention on the HWX Stack
Open Source Bulk Data Loader
FINRA Open Source Libraries for Big Data Testing Hive Scripts in Your IDE
Example) ( GitHub) from Berkley Labs allows for queries on compressed data in Spark.
TensorFlow for Speech Recognition