This is pretty close to a Raspberry Pi Zero. This inexpensive box is another useful IoT device for ingesting some data very inexpensively.
Machine Browsing
root@NanoPi-Duo:/home/pi# cpu_freq INFO: HARDWARE=sun8i CPU0 online=1 temp=48092 governor=ondemand cur_freq=312000 CPU1 online=1 temp=48092 governor=ondemand cur_freq=312000 CPU2 online=1 temp=48092 governor=ondemand cur_freq=312000 CPU3 online=1 temp=48092 governor=ondemand cur_freq=312000
Source Code for Python and Shell Script
https://github.com/tspannhw/nifi-nanopi-duo
Hardware
The FA-CAM202 is a 200M USB camera.
512MB RAM
Ubuntu 16.04.3 LTS 4.11.2
Similar SBC to Raspberry PI
ARM
Software Setup
sudo apt-get install fswebcam -y
sudo apt-get install libv4l-dev -y
sudo apt-get install python-opencv -y
sudo npi-config
sudo apt-get update
sudo apt-get install libcv-dev libopencv-dev -y
pip install psutil
pip2 install psutil
curl http://192.168.1.193:8080/nifi-api/site-to-site/ -v
inferred.avro.schema
{ "type" : "record", "name" : "NANO", "fields" : [ { "name" : "diskfree", "type" : "string", "doc" : "Type inferred from '\"23329.1 MB\"'" }, { "name" : "cputemp", "type" : "double", "doc" : "Type inferred from '55.0'" }, { "name" : "host", "type" : "string", "doc" : "Type inferred from '\"NanoPi-Duo\"'" }, { "name" : "endtime", "type" : "string", "doc" : "Type inferred from '\"2018-01-04 20:23:56\"'" }, { "name" : "ipaddress", "type" : "string", "doc" : "Type inferred from '\"192.168.1.191\"'" }, { "name" : "h", "type" : "int", "doc" : "Type inferred from '342'" }, { "name" : "ts", "type" : "string", "doc" : "Type inferred from '\"2018-01-04 20:23:43\"'" }, { "name" : "filename", "type" : "string", "doc" : "Type inferred from '\"/opt/demo/images/2018-01-04_2023.jpg.faces.jpg\"'" }, { "name" : "w", "type" : "int", "doc" : "Type inferred from '342'" }, { "name" : "memory", "type" : "double", "doc" : "Type inferred from '60.1'" }, { "name" : "y", "type" : "int", "doc" : "Type inferred from '264'" }, { "name" : "x", "type" : "int", "doc" : "Type inferred from '877'" } ] }
hive.ddl
CREATE EXTERNAL TABLE IF NOT EXISTS nanopi (diskfree STRING, cputemp DOUBLE, host STRING, endtime STRING, ipaddress STRING, h INT, ts STRING, filename STRING, w INT, memory DOUBLE, y INT, x INT) STORED AS ORC LOCATION '/nano'
Apache NiFi
The flow in Apache NiFi is pretty simple. We receive the flowfiles from the remote Apache MiniFi box.
1. RouteOnAttribute: Send images to the file system, continue processing the JSON
2. AttributeCleanerProcessor: Clean up the attributes Not really needed for this dataset.
3. UpdateAttribute: Set the schema name to reference the registry
4. SplitJSON: Split JSON into one array per flowfile
5. ConvertRecord: Convert JSON Tree to AVRO with embedded schema
6. ConvertAvroToORC: Build an Apache ORC file (we could add a step before for MergeContent)
7. PutHDFS: Store in Hadoop File System forever
Apache MiniFi
We have a simple flow in Apache MiniFi.
1. ExecuteProcess: Run a shell script to grab a timestamp filenamed image from the USB webcam.
Then call a Python script that does OpenCV Face Detection and adds some local variables to a JSON array with facial squares.
2. GetFile: retrieve all the images from the box and send them to Apache NiFi.
Example Data
[{"diskfree": "23445.5 MB", "cputemp": 56.7, "host": "NanoPi-Duo", "endtime": "2018-01-04 17:40:30", "ipaddress": "192.168.1.191", "h": 55, "ts": "2018-01-04 17:40:20", "filename": "/opt/demo/images/2018-01-04_1740.jpg.faces.jpg", "w": 55, "memory": 22.6, "y": 471, "x": 270}, {"diskfree": "23445.5 MB", "cputemp": 56.7, "host": "NanoPi-Duo", "endtime": "2018-01-04 17:40:30", "ipaddress": "192.168.1.191", "h": 67, "ts": "2018-01-04 17:40:20", "filename": "/opt/demo/images/2018-01-04_1740.jpg.faces.jpg", "w": 67, "memory": 22.6, "y": 625, "x": 464}]
Resources
- http://wiki.friendlyarm.com/wiki/index.php/NanoPi_Duo
- https://diyprojects.io/orange-pi-armbian-control-camera-python-opencv/#.Wku2kFQ-f1J
- https://www.armbian.com/nanopi-duo/
- https://github.com/opencv/opencv.git
- https://github.com/informramiz/Face-Detection-OpenCV
- https://realpython.com/blog/python/face-detection-in-python-using-a-webcam/
- https://realpython.com/blog/python/face-recognition-with-python/
Face
