Community Articles

TimothySpann · ‎02-13-2017

Overview

I have been running a similar program on Raspberry Pi devices with TensorFlow. Now that MXNet has entered Apache incubation, it has become incredibly interesting to me. With the backing of Apache and Amazon, this library cannot be ignored. So I tried in on the same Raspberry Pi 3B that I was using for TensorFlow. For this example, we are grabbing images from the standard Raspberry Pi Camera and running live image analysis on it with MXNet using the Inception pre-built model from the MXNet Model Zoo. This is the nearly the same as the TensorFlow example. What I noticed is a bit faster execution and smoother process. For accuracy, I have not run enough tests for weighing the two libraries out, but that is something I will look at doing for large number of images. Training both with my camera and images I am interested in would be very helpful. Some use cases I am thinking of are: Security Camera, Water Leak Detection, Evil Cat Sensing, Engine Vibration and self-driving model car.

Raspberry Pi v3 B with PI Camera

Setup Your Device For Running MXNet

sudo apt-get -y install git cmake build-essential g++-4.8 c++-4.8 liblapack* libblas* libopencv*
git clone https://github.com/dmlc/mxnet.git --recursive
cd mxnet
make
cd python
sudo python setup.py install
curl --header 'Host: data.mxnet.io' --header 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Firefox/45.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' --header 'Accept-Language: en-US,en;q=0.5' --header 'Referer: http://data.mxnet.io/models/imagenet/' --header 'Connection: keep-alive' 'http://data.mxnet.io/models/imagenet/inception-bn.tar.gz' -o 'inception-bn.tar.gz' -L
tar -xvzf inception-bn.tar.gz
mv Inception_BN-0126.params Inception_BN-0000.params

The primary code is Python taken from some examples from MXNet, OpenCV and PICamera.

topn = inception_predict.predict_from_local_file(filename, N=5)

This calls the inception_predict from MXNet example. The inception_predict code is referenced in the reference links below.

Main Python Code

#!/usr/bin/python
# 2017 load pictures and analyze
import time
import sys
import datetime
import subprocess
import sys
import urllib2
import os
import datetime
import ftplib
import traceback
import math
import random, string
import base64
import json
import paho.mqtt.client as mqtt
import picamera
from time import sleep
from time import gmtime, strftime
import inception_predict

packet_size=3000

def randomword(length):
 return ''.join(random.choice(string.lowercase) for i in range(length))

# Create camera interface
camera = picamera.PiCamera()
while True:

    # Create unique image name
 uniqueid = 'mxnet_uuid_{0}_{1}'.format(randomword(3),strftime("%Y%m%d%H%M%S",gmtime()))

    # Take the jpg image from camera
    filename = '/home/pi/cap.jpg'
    
    # Capture image from RPI
    camera.capture(filename)
 
    # Run inception prediction on image
    topn = inception_predict.predict_from_local_file(filename, N=5)
      
    # CPU Temp
    p = subprocess.Popen(['/opt/vc/bin/vcgencmd','measure_temp'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    out, err = p.communicate()
        
    # MQTT
 client = mqtt.Client()
 client.username_pw_set("username","password")
 client.connect("mqttcloudprovider", 14162, 60)
 
 # CPU Temp
 out = out.replace('\n','')
 out = out.replace('temp=','')
 
 # 5 MXNET Analysis
 top1 = str(topn[0][1])
 top1pct = str(round(topn[0][0],3) * 100)

 top2 = str(topn[1][1])
 top2pct = str(round(topn[1][0],3) * 100)
  
 top3 = str(topn[2][1])
 top3pct = str(round(topn[2][0],3) * 100)

 top4 = str(topn[3][1])
 top4pct = str(round(topn[3][0],3) * 100)

 top5 = str(topn[4][1])
 top5pct = str(round(topn[4][0],3) * 100)
 
 row = [ { 'uuid': uniqueid,  'top1pct': top1pct, 'top1': top1, 'top2pct': top2pct, 'top2': top2,'top3pct': top3pct, 'top3': top3,'top4pct': top4pct,'top4': top4, 'top5pct': top5pct,'top5': top5, 'cputemp': out} ]
 json_string = json.dumps(row)
 
 client.publish("mxnet",payload=json_string,qos=1,retain=False)
 client.disconnect()

We grab an image from a camera, run it through MXNet, convert the results to JSON and then send the message to a cloud hosted MQTT broker. I also grab the CPU temperature to show we can add more sensors.

Example JSON Sent via MQTT

[{"top1pct": "54.5", "top5": "n04590129 window shade", "top4": "n03452741 grand piano, grand", "top3": "n03018349 china cabinet, china closet", "top2": "n03201208 dining table, board", "top1": "n04099969 rocking chair, rocker", "top2pct": "9.1", "top3pct": "8.0", "uuid": "mxnet_uuid_oqy_20170211203727", "top4pct": "2.8", "top5pct": "2.2", "cputemp": "75.2'C"}]

Our schema is pretty consistent as above, so we can create a Hive or Phoenix table and insert into that.

HDF / NiFi Flow

Consume MQTT

This processor will receive messages from a cloud based MQTT broker sent by a few Raspberry PIs I have setup.

Extract Fields from MXNET (EvaluateJSONPath)

Build a Message (UpdateAttribute)

Category 1 ${top1} at ${top1pct}%
Category 2 ${top2} at ${top2pct}%
Category 3 ${top3} at ${top3pct}%
Category 4 ${top4} at ${top4pct}%
Category 5 ${top5} at ${top5pct}%
UUID      ${uuid}
CPU Temp  ${cputemp}

Send Msg to Slack Channel (PutSlack) Channel is mxnet

Stores Files (PutFile)

We take the JSON convert it to a text message to a Slack channel.

That's all it takes to ingest data from an edge device running a camera and running Deep Learning on a tiny device and then send the data asynchronously to a cloud hosted broker that can distribute to cloud and on-premise hosted Apache NiFi servers. We could also use Site-to-Site, HTTP or TCP/IP. MQTT is very lightweight, works over the Internet, has an easy Python library and works well with Apache NiFi.

Reference:

This sample program is critical and gave me most of the code needed to run: http://mxnet.io/tutorials/embedded/wine_detector.html

http://data.mxnet.io/models/imagenet/

https://community.hortonworks.com/content/repo/77987/rpi-picamera-mqtt-nifi.html

https://github.com/tspannhw/mxnet_rpi/blob/master/analyze.py

https://community.hortonworks.com/content/kbentry/80339/iot-capturing-photos-and-analyzing-the-image...

CloudMQTT has proven to be awesome. Instant setup and a free instance for testing. This is great for getting data from my remote raspberry pis to the cloud and back into HDF 2.1 servers behind firewalls. http://cloudmqtt.com

http://www.jsonpath.com/

Github Repo

https://github.com/tspannhw/mxnet_rpi

https://community.hortonworks.com/repos/83001/python-mxnet-raspberry-pi-example.html?shortDescriptio...

Pushing to Slack Channel

https://nifi-se.slack.com/messages/mxnet/details/

Apache MXNet Incubation

https://wiki.apache.org/incubator/MXNetProposal

Awesome MXNet

https://github.com/dmlc/mxnet/tree/master/example

Install MXNet on Raspian

http://mxnet.io/get_started/raspbian_setup.html