This Week in Hadoop and More: Deep Learning and Tools

DZone 's Guide to

This Week in Hadoop and More: Deep Learning and Tools

The world of Hadoop this week had some major updates on Deep Learning, Hadoop, Spark and AI.

· Big Data Zone ·
Free Resource

Security is always a big issue, I wrote a few articles about this recently as people are Google Dorking and using Shodan to find vulnerable, unprotected NoSQL stores like MongoDB, Redis, ElasticSearch, and WebHDFS to attack. Check this out.

Old School Production Grade Mature Hadoop Tools keep coming up, here's a great article on SQOOP Performance. SQOOP is a powerful tool for ingres and egress of data into and out of RDBMS' like Oracle and Postgresql. Hard to beat a tool that can harness a whole cluster for data loading.

A nice article on working with Apache HAWQ (HDB) within Hortonworks HDP. A number of great presentations came out of the AI Frontiers conference on Deep Learning.

This week a few software updates of note, Apache Beam is out of incubation. And so is Apache Ranger. Hive as a Service on CloudFoundry is a very interesting project to try. This github repo has a fun Deep Learning application to manipulate images. On my GPU-less machine it took over an hour to run, so not for real-time streams unless you've pretrained and you have some GPU heavy clusters.

Top Articles of the Week

Top Presentations of the Week

Apache Phoenix and Zeppelin Formatting Hack of the Week


select '%html @' || handle || ' (' || friends_count || ',' || 
followers_count || ',' || retweet_count || 
')  <img width=100 height=100 src="' || 
COALESCE( profile_image_url, 
         'https://upload.wikimedia.org/wikipedia/en/4/48/Blank.JPG') || 
         '"> ' || msg || ' @ ' || location || ' <br> Sentiment: ' 
         || stanfordsentiment || 
' <img width=50 height=50 src="' || 
COALESCE( media_url, 'https://upload.wikimedia.org/wikipedia/en/4/48/Blank.JPG')
|| ' "> '   
as Tweets 
from TweetTable 
where  language = 'en'
and location is not null
and msg like '%Water%main%break%'
order by time desc

Wikimedia has a blank image I am using as a placeholder if I have no image from the tweet.   Coalesce is a nice function in Phoenix that displays the second choice if the first one is null.  A handy function for many cases, especially this one when you are constructing HTML on the fly.

big data ,deep learning ,hadoop ,machine learning ,spark

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}