Over a million developers have joined DZone.

Hadoop Summit San Jose: Data Hacks, Demos, and Raspberry Pi

DZone 's Guide to

Hadoop Summit San Jose: Data Hacks, Demos, and Raspberry Pi

Another in the series of interesting demos from Hadoop Summit, this one shows how to interface Apache NiFi and AWS IoT using Raspberry Pis.

· IoT Zone ·
Free Resource

During the third demo of the Data Hacks and Demos session, at Hadoop Summit San Jose, it was audience participation time! Kay Lerch demonstrated how to interact with the audience through specific twitter and SMS messages sent to a specific phone number, or tweet to @HadoopSummitDemo. The messages were routed through to the light boxes, which would flash and light for each message received, a vote for each question and hashtag represented.

So what did Joe and Kay tell the audience?

This demo simulated how a customer can share their real-time feedback with the retailer about their preferences and opinions. In real life, consumers may express their opinion digitally, (i.e. on social media such as Twitter) rather than providing feedback in the brick and mortar store. Being able to digitally interact with them and understand their experience provides timely information that can influence their purchasing decision.

In this case, the audience was asked to vote on this demo:

  • A) Does this demo make you more excited about your future experience as a consumer – that a retailer could know you better? If so, vote “More excited.”
  • Or

  • B) does this demo his make you nervous with respect to its privacy implications? If so, vote “More Concerned.”
  • Slide1

    As it turned out, the audience was about evenly split between nervous or excited about such tech. And in the end, the system processed almost 3,000 votes in total in fewer than three minutes, after which, the batteries for lighting up the LED lightboxes were exhausted!

    Demo 3 AWS IoT Apache NiFi Live Voting Hack Demos Hortonworks HS16SJ

    How Did it Work?

    Behind the scenes, there was an Apache NiFi instance hosted on Amazon EC2 orchestrating a dataflow. Data was ingested from Twilio (an SMS provider) and the Twitter Streaming API. The Apache NiFi instance parsed the information and passed the vote over to AWS IoT.

    AWS IoT was used as an IoT Gateway to communicate over MQTT with the two Raspberry Pis sitting inside the Lightboxes. These Raspberry Pis controlled the 32×32 LED panels. The lightboxes interpreted incoming SMS message as red flash and an incoming tweet as a green flash.

    Also connected to the Raspberry Pi were temperature and humidity sensors. The data collected by these sensors is sent back to AWS IoT and stored in the so-called “thing shadow”.  The shadow was then read out by NiFi for each one of your messages.

    This sensor data was sent back the same way back to the mobile phone the way the vote information was delivered to the lightboxes. The Raspberry Pi sends the sensor data to AWS IoT over MQTT, persisted in a thing shadow of AWS IoT, then processed by NiFi and returned to Twilio in response to its webhook resulting in instant real-time feedback as a response SMS.

    That was the third demo of Data Hacks & Demos at Hadoop Summit San Jose. The fourth demo — using streaming analytics to create a single buyer identity — is up next in this blog series.

    iot ,hortonworks ,nifi ,aws iot ,cloud ,raspberry pi

    Published at DZone with permission of

    Opinions expressed by DZone contributors are their own.

    {{ parent.title || parent.header.title}}

    {{ parent.tldr }}

    {{ parent.urlSource.name }}