Analyzing Images in HDF 2.0 Using Tensorflow
Take a look at using Google's TensorFlow Deep Learning library for identifying images taken in drones. There is even a NiFi programming example provided.
Join the DZone community and get the full member experience.Join For Free
As part of a live Drone ingest, I want to identify what in the image. The metadata provides a ton of information on GPS, altitude and image characteristics, but not what's in the image.
IBM, Microsoft, and Google all have APIs that do a good job of this and they for the most part of "free" tiers. I wanted to run something locally using libraries installed on my cluster. For my first option, I used TensorFlow Inception-v3 Image Recognition. In future articles, I will cover PaddlePaddle, OpenCV and some other Deep Learning and non-deep learning options for Image Recognition. Also, I will show the entire Drone to Front-End flow including Phoenix, Spring Boot, Zeppelin, LeafletJS and more. This will be done as part of a meetup presentation with a certified drone pilot.
To Run My TensorFlow Binary From HDF 2.0 I use the ExecuteStreamCommand to run a shell script containing the information below:
source /usr/local/lib/bazel/bin/bazel-complete.bash export JAVA_HOME=/opt/jdk1.8.0_101/ hdfs dfs -get /drone/raw/$@ /tmp/ /opt/demo/tensorflow/bazel-bin/tensorflow/examples/label_image/label_image --image="/tmp/$@" --output_layer="softmax:0" --input_layer="Mul:0" --input_std=128 --input_mean=128 --graph=/opt/demo/tensorflow/tensorflow/examples/label_image/data/tensorflow_inception_graph.pb --labels=/opt/demo/tensorflow/tensorflow/examples/label_image/data/imagenet_comp_graph_label_strings.txt 2>&1| cut -c48-
In my script, I pull the file out of HDFS (that was loaded by HDF 2.0) and then run the binary version of TensorFlow that I compiled for Centos7. If you can't or don't want to install Bezel and build that, then you can run the Python script, it's a little bit slower and has slightly different output.
python /usr/lib/python2.7/site-packages/tensorflow/models/image/imagenet/classify_image.py --image_file
The C++ version that I compiled is Google's example and you can take a look at it:
It's very clean code if you wish to tweak it.
You must have JDK 1.8 (and know the path), not just JRE 1.8. Also, you need Python 2.7 or 3.+ and PIP. You need to install Google's Bezel build tool.
sudo yum groupinstall "Development Tools" sudo yum install gettext-devel openssl-devel perl-CPAN perl-devel zlib-develsudo yum -y install epel_release sudo yum -y install gcc gcc-c++ python-pip python-devel atlas atlas-devel gcc-gfortran openssl-devel libffi-devel pip install --upgrade numpy scipy wheel cryptography export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0-cp27-none-linux_x86_64.whl pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0-cp27-none-linux_x86_64.whl git clone https://github.com/tensorflow/tensorflow cd tensorflow/ wget https://storage.googleapis.com/download.tensorflow.org/models/inception_dec_2015.zip -O tensorflow/examples/label_image/data/inception_dec_2015.zip unzip tensorflow/examples/label_image/data/inception_dec_2015.zip -d tensorflow/examples/label_image/data/ cd tensorflow/examples/label_image /opt/demo/bazel/output/bazel label_image
Run Results of TensorFlow
python classify_image.py --image_file /opt/demo/dronedataold/Bebop2_20160920083655-0400.jpg solar dish, solar collector, solar furnace (score = 0.98316) window screen (score = 0.00196) manhole cover (score = 0.00070) radiator (score = 0.00041) doormat, welcome mat (score = 0.00041) bazel-bin/tensorflow/examples/label_image/label_image --image=/opt/demo/dronedataold/Bebop2_20160920083655-0400.jpg solar dish (577): 0.983162I window screen (912): 0.00196204I manhole cover (763): 0.000704005I radiator (571): 0.000408321I doormat (972): 0.000406186
The image is a picture of solar panels on a residential black tar roof.
Published at DZone with permission of Tim Spann, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.