OpenCV + Apache MiniFi for IoT

Let's learn how to ingest camera data from a NanoPi Duo with a helping hand from Apache MiniFi, Hadoop, and some Python. We'll then add image recognition through OpenCV Face Detection.

Let's start by examining our NanoPi Duo.

This is pretty close to a Raspberry Pi Zero. This inexpensive box is another useful IoT device for ingesting some data very inexpensively.

Machine Browsing

root@NanoPi-Duo:/home/pi# cpu_freq
INFO: HARDWARE=sun8i
CPU0 online=1 temp=48092 governor=ondemand cur_freq=312000
CPU1 online=1 temp=48092 governor=ondemand cur_freq=312000
CPU2 online=1 temp=48092 governor=ondemand cur_freq=312000
CPU3 online=1 temp=48092 governor=ondemand cur_freq=312000

The source code for the Python and shell script we'll be using can be found here.

Hardware

• The FA-CAM202 is a 200M USB camera.

• 512MB RAM

• Ubuntu 16.04.3 LTS 4.11.2

• Similar SBC to Raspberry Pi

• ARM

Software Setup

sudo apt-get install fswebcam -y

sudo apt-get install libv4l-dev -y

sudo apt-get install python-opencv -y

sudo npi-config

sudo apt-get update

sudo apt-get install libcv-dev libopencv-dev -y

pip install psutil

pip2 install psutil

curl http://192.168.1.193:8080/nifi-api/site-to-site/ -v

inferred.avro.schema

{
	"type": "record",
	"name": "NANO",
	"fields": [
		{
			"name": "diskfree",
			"type": "string",
			"doc": "Type inferred from '\"23329.1 MB\"'"
		},
		{
			"name": "cputemp",
			"type": "double",
			"doc": "Type inferred from '55.0'"
		},
		{
			"name": "host",
			"type": "string",
			"doc": "Type inferred from '\"NanoPi-Duo\"'"
		},
		{
			"name": "endtime",
			"type": "string",
			"doc": "Type inferred from '\"2018-01-04 20:23:56\"'"
		},
		{
			"name": "ipaddress",
			"type": "string",
			"doc": "Type inferred from '\"192.168.1.191\"'"
		},
		{
			"name": "h",
			"type": "int",
			"doc": "Type inferred from '342'"
		},
		{
			"name": "ts",
			"type": "string",
			"doc": "Type inferred from '\"2018-01-04 20:23:43\"'"
		},
		{
			"name": "filename",
			"type": "string",
			"doc": "Type inferred from '\"/opt/demo/images/2018-01-04_2023.jpg.faces.jpg\"'"
		},
		{
			"name": "w",
			"type": "int",
			"doc": "Type inferred from '342'"
		},
		{
			"name": "memory",
			"type": "double",
			"doc": "Type inferred from '60.1'"
		},
		{
			"name": "y",
			"type": "int",
			"doc": "Type inferred from '264'"
		},
		{
			"name": "x",
			"type": "int",
			"doc": "Type inferred from '877'"
		}
	]
}

hive.ddl

CREATE external TABLE 
  IF NOT EXISTS nanopi (diskfree string, cputemp DOUBLE, host string, endtime string, ipaddress string, h INT, ts string, filename string, w INT, memory DOUBLE, y INT, x INT) stored
  AS 
    orc location '/nano'

Apache NiFi

The flow in Apache NiFi is pretty simple. We receive the flowfiles from the remote Apache MiniFi box.

1. RouteOnAttribute: Send images to the file system, continue processing the JSON

2. AttributeCleanerProcessor: Clean up the attributes not really needed for this dataset.

3. UpdateAttribute: Set the schema name to reference the registry

4. SplitJSON: Split JSON into one array per flowfile

5. ConvertRecord: Convert JSON tree to AVRO with embedded schema

6. ConvertAvroToORC: Build an Apache ORC file (we could add a step before for MergeContent)

7. PutHDFS: Store in Hadoop File System forever

Apache MiniFi

We have a simple flow in Apache MiniFi.

1. ExecuteProcess: run a shell script to grab a timestamp filenamed image from the USB webcam. Then call a Python script that does OpenCV Face Detection and adds some local variables to a JSON array with facial squares.

2. GetFile: retrieve all the images from the box and send them to Apache NiFi.

Example Data

[
	{
		"diskfree": "23445.5 MB",
		"cputemp": 56.7,
		"host": "NanoPi-Duo",
		"endtime": "2018-01-04 17:40:30",
		"ipaddress": "192.168.1.191",
		"h": 55,
		"ts": "2018-01-04 17:40:20",
		"filename": "/opt/demo/images/2018-01-04_1740.jpg.faces.jpg",
		"w": 55,
		"memory": 22.6,
		"y": 471,
		"x": 270
	},
	{
		"diskfree": "23445.5 MB",
		"cputemp": 56.7,
		"host": "NanoPi-Duo",
		"endtime": "2018-01-04 17:40:30",
		"ipaddress": "192.168.1.191",
		"h": 67,
		"ts": "2018-01-04 17:40:20",
		"filename": "/opt/demo/images/2018-01-04_1740.jpg.faces.jpg",
		"w": 67,
		"memory": 22.6,
		"y": 625,
		"x": 464
	}
]

Face