In preparation for my talk at the Philadelphia Open Source Conference, Apache Deep Learning 201, I wanted to have some good images for running various Apache MXNet Deep Learning Algorithms for Computer Vision. 

Using Apache open source tools - Apache NiFi 1.8 and Apache MXNet 1.3 with GluonCV I can easily ingest live traffic camera images and run Object Detection, Semantic Segmentation and Instance Segmentation.

Code: (Github)

It's so easy, I am running multiple on the data to see which gives me the results I like. I am like YOLO, which is one of my old favorites.

• yolonifitraffic.py
• demo_mask_rcnn_nyc.py
• demo_deeplab_nyc.py

So, we can find the cars and people in these webcams. Use cases can be around traffic optimization, public safety, and advertisement optimization. Due to licensing, I thought it better not to show Traffic Camera data here.

To industrialize and scale out this process from a single Data Scientist to a national ingestion system, we use the power of Apache NiFi to ingest, process and control flows. I am using the latest Apache NiFi 1.8.

Apache NiFi Flow to Ingest and Process Traffic Camera Data

First, we have a list of URLs that I want to process, and this can be sourced and stored anywhere. For ease of use with a static set, I am using GenerateFlowFile. I have a JSON file of URLs that I split and parse to call various Computer Vision Python scripts (DeepLab3, MaskRCNN, YOLO and others). YOLO seems to be the most useful so far. I am grabbing the results, some system metrics, metadata, and the Deep Learning analytics generated by Apache MXNet.

I split the flow into two portions. One builds GluonCV result data from YOLO and the other creates a file from TensorFlow results done on the fly.

Here is a list of my webcam URLs. There are millions of them out there.

If your data is tabular, then you need a schema for fast record processing.

An Example Dataset Returned from GLUONCV — YOLO Python 3.6 Script

I turn JSON data into HDFS Writeable AVRO Data and can run live SQL on it

One Output Source Code Be a Joint Slack Group

Object Detection: GluonCV YOLO v3 and Apache NiFi

This can be OpenCV, a static photo or from a URL.

Object Detection: Faster RCNN with GluonCV

Faster RCNN model trained on Pascal VOC dataset with ResNet-50 backbone

net = gcv.model_zoo.get_model(faster_rcnn_resnet50_v1b_voc, pretrained=True)

Instance Segmentation: Mask RCNN with GluonCV

Mask RCNN model trained on COCO dataset with ResNet-50 backbone

net = model_zoo.get_model('mask_rcnn_resnet50_v1b_coco', pretrained=True)

Photo by Ryoji Iwata on Unsplash

There are a lot of people crossing the street!

Semantic Segmentation: DeepLabV3 with GluonCV

GluonCV DeepLabV3 model on ADE20K dataset

model = gluoncv.model_zoo.get_model('deeplab_resnet101_ade', pretrained=True)

run1.sh demo_deeplab_webcam.py

This runs pretty slow on a machine with no GPU.

That is the best picture of me ever!

Semantic Segmentation: Fully Convolutional Networks

GluonCV FCN model on PASCAL VOC dataset

model = gluoncv.model_zoo.get_model(‘fcn_resnet101_voc ', pretrained=True)

run1.sh demo_fcn_webcam.py

It found me.

For NYC Dot and PennDot camera usage, you have to sign a developer agreement for a feed!