Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Integrating TensorFlow 1.6 Image Labelling with HDF 3.1 and Apache NiFi 1.5

DZone's Guide to

Integrating TensorFlow 1.6 Image Labelling with HDF 3.1 and Apache NiFi 1.5

A data scientist gives a tutorial on how to use the TensorFlow AI library to help with image analysis as part of an Apache NiFi flow.

· Integration Zone ·
Free Resource

SnapLogic is the leading self-service enterprise-grade integration platform. Download the 2018 GartnerMagic Quadrant for Enterprise iPaaS or play around on the platform, risk free, for 30 days.

Integrating TensorFlow 1.6 Image Labelling with HDF 3.1 and Apache NiFi 1.5

This is community Apache NiFi custom processor that I have written with help from Simon Ball. This is after his original change, I converted the results to a different style and incorporated changes to Google's TensorFlow Label_Image in a Java example.

Images Supported as Flow Files

JPG, PNG, GIF

Versions

Updated TensorFlowProcessor to TF 1.6.

Summary

I added more tests, did more code cleanup, changed the top 5 returned with cleaner naming. This processor has been tested in a few environments, please give it a try and let me know your results. The performance and stability seem quite good, hopefully enough for your use cases. You can also use the MiniFi C++ Agent's built-in C++ TensorFlow support if you wish to have fast local TensorFlow at the edge.

Video

JUNit Test for TensorFlow Processor

Here is another flow where I am sending an image to two Apache MXNet Model Servers and the local TensorFlow processor.

This is an example of the attributes returned by the processor. You pass in the image as a flow file and it adds attributes without changing the contents.

In Apache NiFi, we can monitor how long the tasks are taking. After the first run, the time of the processor goes down as we have cached the labels and the TensorFlow pre-built graph.

This is an example of an actual flow. I use this to grab images from Twitter, download them and run TensorFlow on them.

Installation

  1. Download NAR herehttps://github.com/tspannhw/nifi-tensorflow-processor/releases/tag/1.6
  2. Install nar file to /usr/hdf/current/nifi/lib/
  3. Create a model directory
  4. wget https://raw.githubusercontent.com/tspannhw/nifi-tensorflow-processor/master/nifi-tensorflow-processors/src/test/resources/models/imagenet_comp_graph_label_strings.txt
  5. wget https://github.com/tspannhw/nifi-tensorflow-processor/blob/master/nifi-tensorflow-processors/src/test/resources/models/tensorflow_inception_graph.pb?raw=true
  6. Restart Apache NiFi via Ambari

Download A Buyer's Guide to Application and Data Integration, your one-stop-shop for research, checklists, and explanations for an application and data integration solution.

Topics:
hadoop ,apache nifi ,tensorflow ,integration

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}