Combining DJL.AI With Apache NiFi for Deep Learning Workflows

NiFi + DJL.AI = A Merry Deep Learning Christmas

Happy Mmm...FLaNK Day!

I have been experimenting with the awesome new Apache 2.0 licensed Java Deep Learning Library, DJL. In NiFi, I was trying to figure out a quick use case and demo. So I used my Web Camera processor to grab a still shot from my Powerbook webcam and send it to the processor. The results are sent to slack.

Since it's the holidays, I think of my favorite holiday movies: The Matrix and Blade Runner. So I thought a Voight-Kampf test would be fun. Since I don't have a deep learning QA piece built yet, let's start by seeing if you look human. We'll call them 'person'. I am testing to see if I am a replicant. Sometimes it's hard to tell. Let's see if DJL thinks I am human.

See: http://nautil.us/blog/-the-science-behind-blade-runners-voight_kampff-test

Okay, so at least it thinks I am a person. The classification of a Christmas tree is vaguely accurate. So we can see how easy and powerful Deep Learning is in Java, adding it to NiFi makes it even easier.

It did not identify my giant french bread.

You might also be interested in:  Using Apache Spark and Apache NiFi to Run TensorFlow

Building a New Custom Processor for Deep Learning

The hardest part was a good NiFi Integration test. The DJL team provides some great examples, and it's really easy to plug into their models.

xxxxxxxxxx

ZooModel<BufferedImage, DetectedObjects> model =

                     MxModelZoo.SSD.loadModel(criteria, new ProgressBar())

Predictor<BufferedImage, DetectedObjects> predictor = model.newPredictor()

DetectedObjects detection = predictor.predict(img);

All the sources are on GitHub and reference the below DJL sites and repos.

Using a New Custom Processor as Part of a Real-time Holiday Flow

We first add the DeepLearningProcessor to our canvas.

An example flow:

• GetWebCameraProcessor: grab an image from an attached webcamera
• UpdateAttribute: Add media type for image
• DeepLearningProcessor: Run our DJL deep learning model from a zoo
• PutSlack: Put DJL results in a text window in slack
• PostSlack: Send our DJL altered image to slack
• Funnel: Send all failures to Valhalla

If we example the provenance, we can see how long it took to run and some other interesting attributes.

We place the results of our image analysis in attributes while we return a new image that has a bounding box on the found object(s).

We now a full classification workflow for real-time deep learning analysis on images, could be used for Santa watching, Security, Memes and other important business purposes.

The initial release is available here: https://github.com/tspannhw/nifi-djl-processor/releases/tag/v1.0

Using library and example code from the Deep Java Library (https://djl.ai/).

• https://github.com/awslabs/djl#getting-started

• https://djl.ai/mxnet/mxnet-model-zoo/

• https://github.com/awslabs/djl/blob/master/examples/pom.xml

• https://github.com/tspannhw/djl/blob/master/examples/src/main/java/ai/djl/examples/inference/ObjectDetection.java

• https://github.com/aws-samples/djl-demo

• https://github.com/awslabs/djl/blob/master/jupyter/BERTQA.ipynb

• https://github.com/awslabs/djl/blob/master/examples/src/main/java/ai/djl/examples/inference/BertQaInference.java

Source Code: https://github.com/tspannhw/nifi-djl-processor/

Further Reading

Exploring Apache NiFi 1.10: Parameters and Stateless Engine

Apache NiFi Overview