Over a million developers have joined DZone.

This Week in Hadoop and More: NiFi, CarbonData, and More

DZone's Guide to

This Week in Hadoop and More: NiFi, CarbonData, and More

The best new talks from around the world available this week including talks on Hadoop, NiFi, Hive, and Spark from Hadoop Summit Melbourne.

· Big Data Zone ·
Free Resource

The open source HPCC Systems platform is a proven, easy to use solution for managing data at scale. Visit our Easy Guide to learn more about this completely free platform, test drive some code in the online Playground, and get started today.

There are a few newer projects that are of interest including Apache CarbonData as a new Columnar Data Format for fast queries. 

Apache CarbonData is yet another sub-second response column format available on Github and Incubator. Check out this summary of what CarbonData is. It is from Huawei who are doing some really interesting Big Data and Spark work.

Astro, Another SQL Interface to HBase (on Spark), also from Huawei. Another one to watch, I am hoping to do a benchmark to see how it compares to Phoenix. I like Phoenix since it can work from regular ODBC/JDBC and doesn't require Spark. Though a day without Spark is not a happy one. If I don't have some Spark and NiFi daily, I think my pipeline is broken. I would like to see Astro more generic, maybe on Apache Beam.

Some decent introduction to Core Hadoop: Introduction to Scheduling in Hadoop and Introduction  to Yarn (Apex as a Yarn App).

If you want to elastically scale a small Hadoop/Spark cluster on Amazon, then check out this awesome free tool to quickly Run Open Source Hadoop Distribution on AWS.

This week's most interesting articles span NiFi, SAP Hana, Heron, Kafka, and Hadoop.

Hadoop Summit Melbourne 2016

Managing data at scale doesn’t have to be hard. Find out how the completely free, open source HPCC Systems platform makes it easier to update, easier to program, easier to integrate data, and easier to manage clusters. Download and get started today.

hadoop ,big data ,hortonworks ,spark ,sql ,zeppelin ,nifi ,yarn ,hbase ,cloud

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}