This Week in Hadoop and More: Global Presentations

There's a great deal of content from Spark, Flink, Hadoop, and NiFi from conferences around the world. Also, we introduce a few other great new resources.

Tim Spann

CORE ·

Nov. 18, 16 · News

Like (3)

Save

3.48K Views

FlinkForward Conference 2016

My friend Julian Hyde does a very interesting talk on StreamingSQL which is going to be huge in many of the streaming systems. With Apache Calcite, the hope is that all the different Streaming SQLs will be using the same tools, syntax and libraries so we can have compatibility across streaming libraries.

Another sign that Apache Flink is becoming more mature, Apache Flink Security Enhancements.

One of the interesting features of Flink is Code Generation in Serializers and Comparators.

Flink is now able to work with Apache Zeppelin, making on par with the ease of data exploration that you get with Apache Spark.

Spark Summit EU 2016

Spark for ETL (Nov 2016)

Spark and Lucene for near-realtime predictive model building

Problem Solving Recipes Learned from Supporting Spark

Elastic Streaming (Spark)

Sparkling Insights at Elsevier

Spark - Universal Computation Engine for Processing Oil Industry Data

Spark and SAP HANA Vora

Online Learning With Structured Streaming

Adopting Dataframes and Parquet in an already existing datawarehouse

Spark ML with Redis

Sparksheet - Transforming Spreadsheets into Spark Data Frames (GitHub)

Boosting Spark Performance on Many-Core Machines

Performance Characterization of In-Memory Data Analytics on a Scale-up Server

Extending Spark with Java Agents (GitHub)

SparkSQL 2.0/2.1 Experiences using TPC-DS

Apache Spark and Object Stores - What you need to know

Deep Dive into Catalyst: Apache Spark's Optimizer

Apache Ignite with Apache Spark

Sparkling Water 2.0: H2O Deep Water for working with TensorFlow, MXNet, and Caffe and with Zeppelin. This really expands what you can do on the JVM.

Effective Spark with Alluxio

Mastering Spark Unit Testing

Online Structured Streaming

Apache Spark 2.0 Performance Improvements Investigated with Flame Graphs (Details)

Streaming Outlier Analysis for Fun and Scalability

IoT With NIFI

For complex IoT programming, you need a combination of Apache NiFi, Storm, and Kafka Together. This is a great talk on Edge Intelligence IoT with MiniFi. Installing MiniFI on Your Device, will allow you to access all kinds of IoT devices and stream their data into your corporate NIFI servers. MiniFi in C++ - Very early, but fast small C++ compiled code for devices without Java.