This Week in Hadoop (Part I): Spark Uber Kit, Hive 2.1, and More

The big events this week are the Hadoop Summit in San Jose and also MongoDB World in NYC. 

Apache™ Hadoop® distributions products from Altiscale, ArenaData, Hortonworks, IBM, and Infosys are now ODPi Runtime Compliant so that all now comply with base level standards which is great for standardization.   I will be talking to the ODPi in an upcoming Dzone interactive webinar for all those with questions (or leave comments here).

HDP 2.5 release will contain Zeppelin, Apache Atlas and Apache Ranger, a preview of the new version of Spark, improved Ambari, Apache Phoenix, and more.

Download it today, then try the 2.5 tutorials

Some great new presentations on Big Data:

• Dataflow with Apache NiFi (Apache NiFi Meetup)

• Apache Accumulo 1.8.0 Overview by Josh Elser

• Including Mirroring Datasets with Apache Falcon

• Using Airflow for Spark

Also, video of the week so far is Rob Bearden, Hortonworks - Executive On-the-Ground #theCUBE 

Hortonworks, Arizona State University, Baylor College of Medicine, Booz Allen Hamilton, Mayo Clinic, OneOme and Yale New Haven Health have started a Precision Medicine Consortium to work on Open Source Genomics to cure cancer.

Learning Resources:

• Using HDInsight

• Spark Uber Dev Kit

• Organizing Google Datasets

• Hive 2.1 for download

• Airflow Tutorial

• Tech Preview 2 with HDP 2.4 cluster, ElasticSearch 2.3.X and Kibana 4

• Installing Metron on a Cluster

• Setting up a Vagrant Local Development Metron.   Note:  Disable antivirus as it contains items that will set off virus detector.

•  Cool article on database ingest with NiFi