Community Articles

TimothySpann · ‎11-17-2017

For this edge use case we are using NVidia's TensorRT as well as Apache MXNet. From TensorRT I am using imageNet for image recognition and detectNet for object localization.

For Apache MXNet, I am using their image classifier. So we have multiple deep learning frameworks run on the same capture from an attached USB webcam. For this example I am using a Logitech HD1080, while the Jetson TX1 supports 6+ concurrent high end cameras for those with high end use cases. They also have a more powerful Jetson TX2 for more intense use cases as it has more RAM and a better GPU.

Quick Hardware Breakdown

NVIDIA Maxwell™ GPU with 256 NVIDIA® CUDA® Cores
4 GB LPDDR4 Memory

Python Script

# 2017 load pictures and analyze
# https://github.com/tspannhw/mxnet_rpi/blob/master/analyze.py
import time
import sys
import datetime
import subprocess
import sys
import urllib2
import os
import datetime
import traceback
import math
import random, string
import base64
import json
from time import gmtime, strftime
import mxnet as mx
import inception_predict
import numpy as np
import cv2
import math
import random, string
import time
from time import gmtime, strftime
start = time.time()
cap = cv2.VideoCapture(0)
packet_size=3000
def randomword(length):
 return ''.join(random.choice(string.lowercase) for i in range(length))
#while True:
# Create unique image name
uniqueid = 'mxnet_uuid_{0}_{1}'.format(randomword(3),strftime("%Y%m%d%H%M%S",gmtime()))
ret, frame = cap.read()
imgdir = 'images/'
filename = 'tx1_image_{0}_{1}.jpg'.format(randomword(3),strftime("%Y%m%d%H%M%S",gmtime()))
cv2.imwrite(imgdir + filename, frame)
# Run inception prediction on image
try:
     topn = inception_predict.predict_from_local_file(imgdir + filename, N=5)
except:
     errorcondition = "true"
# CPU Temp
f = open("/sys/devices/virtual/thermal/thermal_zone1/temp","r")
cputemp = str( f.readline() )
cputemp = cputemp.replace('\n','')
cputemp = cputemp.strip()
cputemp = str(round(float(cputemp)) / 1000)
cputempf = str(round(9.0/5.0 * float(cputemp) + 32))
f.close()
# GPU Temp
f = open("/sys/devices/virtual/thermal/thermal_zone2/temp","r")
gputemp = str( f.readline() )
gputemp = gputemp.replace('\n','')
gputemp = gputemp.strip()
gputemp = str(round(float(gputemp)) / 1000)
gputempf = str(round(9.0/5.0 * float(gputemp) + 32))
f.close()
# Face Detect
p = os.popen('/media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/jetson-inference-master/build/aarch64/bin/facedetect.sh ' + filename).read()
face = p.replace('\n','|')
face = face.strip()
# NVidia Image Net Classify
p2 = os.popen('/media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/jetson-inference-master/build/aarch64/bin/runclassify.sh ' + filename).read()
imagenet = p2.replace('\n','|')
imagenet = imagenet.strip()
# 5 MXNET Analysis
top1 = str(topn[0][1])
top1pct = str(round(topn[0][0],3) * 100)
top2 = str(topn[1][1])
top2pct = str(round(topn[1][0],3) * 100)
top3 = str(topn[2][1])
top3pct = str(round(topn[2][0],3) * 100)
top4 = str(topn[3][1])
top4pct = str(round(topn[3][0],3) * 100)
top5 = str(topn[4][1])
top5pct = str(round(topn[4][0],3) * 100)
end = time.time()
# face[-4096:]
row = { 'uuid': uniqueid,  'top1pct': top1pct, 'top1': top1, 'top2pct': top2pct, 'top2': top2,'top3pct': top3pct, 'top3': top3,'top4pct': top4pct,'top4': top4, 'top5pct': top5pct,'top5': top5, 'cputemp': cputemp, 'gputemp': gputemp, 'imagefilename': filename, 'gputempf': gputempf, 'cputempf': cputempf, 'runtime': str(round(end - start)), 'facedetect': face, 'imagenet': imagenet }
json_string = json.dumps(row)
print (json_string )

Setup Jetson TX1 for Deep Learning and Computer Vision

sudo apt-get update -y
sudo apt-get -y install git build-essential libatlas-base-dev libopencv-dev graphviz python-pip
sudo pip install pip --upgrade
sudo pip install setuptools numpy --upgrade

Apache Hive DDL

CREATE EXTERNAL TABLE IF NOT EXISTS jetsonscan 
(top3pct STRING, uuid STRING, top1pct STRING, top5 STRING, top4 STRING, top3 STRING, top2 STRING, top1 STRING, top4pct STRING, facedetect STRING, gputempf STRING, gputemp STRING, top5pct STRING, top2pct STRING, cputemp STRING, imagenet STRING, runtime STRING, imagefilename STRING, cputempf STRING) STORED AS ORC
LOCATION '/jetsonscan'

Build Apache MiniFi Configuration

minifi-toolkit-0.2.0/bin/config.sh transform $1 config.yml
scp config.yml nvidia@192.168.1.190:/media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/minifi-0.2.0/conf/

Example Output JSON

{
  "top3pct" : "6.1",
  "uuid" : "mxnet_uuid_pgo_20171110193628",
  "top1pct" : "8.3",
  "top5" : "n03110669 cornet, horn, trumpet, trump",
  "top4" : "n03481172 hammer",
  "top3" : "n02787622 banjo",
  "top2" : "n02791270 barbershop",
  "top1" : "n04487394 trombone",
  "top4pct" : "4.4",
  "facedetect" : "networks/facenet-120/snapshot_iter_24000.caffemodel initialized.|[cuda]  cudaAllocMapped 16 bytes, CPU 0x1013a0000 GPU 0x1013a0000|maximum bounding boxes:  3136|[cuda]  cudaAllocMapped 50176 bytes, CPU 0x1012a6200 GPU 0x1012a6200|[cuda]  cudaAllocMapped 12544 bytes, CPU 0x1011a1a00 GPU 0x1011a1a00|failed to load image /media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/images/tx1_image_xmv_20171110193629.jpg|failed to load image '/media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/images/tx1_image_xmv_20171110193629.jpg'|",
  "gputempf" : "68.0",
  "gputemp" : "20.0",
  "top5pct" : "3.2",
  "top2pct" : "6.4",
  "cputemp" : "21.5",
  "imagenet" : "imagenet-console|  args (3):  0 [/media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/jetson-inference-master/build/aarch64/bin/imagenet-console]  1 [/media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/images/tx1_image_xmv_20171110193629.jpg]  2 [/media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/images/cfout-tx1_image_xmv_20171110193629.jpg]  |||imageNet -- loading classification network model from:|         -- prototxt     networks/googlenet.prototxt|         -- model        networks/bvlc_googlenet.caffemodel|         -- class_labels networks/ilsvrc12_synset_words.txt|         -- input_blob   'data'|         -- output_blob  'prob'|         -- batch_size   2||[GIE]  attempting to open cache file networks/bvlc_googlenet.caffemodel.2.tensorcache|[GIE]  loading network profile from cache... networks/bvlc_googlenet.caffemodel.2.tensorcache|[GIE]  platform has FP16 support.|[GIE]  networks/bvlc_googlenet.caffemodel loaded|[GIE]  CUDA engine context initialized with 2 bindings|[GIE]  networks/bvlc_googlenet.caffemodel input  binding index:  0|[GIE]  networks/bvlc_googlenet.caffemodel input  dims (b=2 c=3 h=224 w=224) size=1204224|[cuda]  cudaAllocMapped 1204224 bytes, CPU 0x100ce0000 GPU 0x100ce0000|[GIE]  networks/bvlc_googlenet.caffemodel output 0 prob  binding index:  1|[GIE]  networks/bvlc_googlenet.caffemodel output 0 prob  dims (b=2 c=1000 h=1 w=1) size=8000|[cuda]  cudaAllocMapped 8000 bytes, CPU 0x100e20000 GPU 0x100e20000|networks/bvlc_googlenet.caffemodel initialized.|[GIE]  networks/bvlc_googlenet.caffemodel loaded|imageNet -- loaded 1000 class info entries|networks/bvlc_googlenet.caffemodel initialized.|failed to load image /media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/images/tx1_image_xmv_20171110193629.jpg|failed to load image '/media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/images/tx1_image_xmv_20171110193629.jpg'|",
  "runtime" : "8.0",
  "imagefilename" : "tx1_image_xmv_20171110193629.jpg",
  "cputempf" : "71.0" 
}

Schema (Put this in Hortonworks Schema Registry) - MXRECORD

{ "type" : "record", "name" : "MXRECORD", "fields" : 
[ { "name" : "top3pct", "type" : "string", "doc" : "Type inferred from '\"5.0\"'" }, 
  { "name" : "uuid", "type" : "string", "doc" : "Type inferred from '\"mxnet_uuid_ltu_20171110193847\"'" }, 
  { "name" : "top1pct", "type" : "string", "doc" : "Type inferred from '\"5.4\"'" }, 
  { "name" : "top5", "type" : "string", "doc" : "Type inferred from '\"n03970156 plunger, plumber's helper\"'" }, 
  { "name" : "top4", "type" : "string", "doc" : "Type inferred from '\"n07615774 ice lolly, lolly, lollipop, popsicle\"'" }, 
  { "name" : "top3", "type" : "string", "doc" : "Type inferred from '\"n04270147 spatula\"'" }, 
  { "name" : "top2", "type" : "string", "doc" : "Type inferred from '\"n03110669 cornet, horn, trumpet, trump\"'" }, 
  { "name" : "top1", "type" : "string", "doc" : "Type inferred from '\"n04487394 trombone\"'" }, 
  { "name" : "top4pct", "type" : "string", "doc" : "Type inferred from '\"4.5\"'" }, 
  { "name" : "facedetect", "type" : "string" }, 
  { "name" : "gputempf", "type" : "string", "doc" : "Type inferred from '\"68.0\"'" }, 
  { "name" : "gputemp", "type" : "string", "doc" : "Type inferred from '\"20.0\"'" }, 
  { "name" : "top5pct", "type" : "string", "doc" : "Type inferred from '\"4.4\"'" }, 
  { "name" : "top2pct", "type" : "string", "doc" : "Type inferred from '\"5.3\"'" }, 
  { "name" : "cputemp", "type" : "string", "doc" : "Type inferred from '\"23.0\"'" }, 
  { "name" : "imagenet", "type" : "string" }, 
  { "name" : "runtime", "type" : "string", "doc" : "Type inferred from '\"8.0\"'" }, 
  { "name" : "imagefilename", "type" : "string", "doc" : "Type inferred from '\"tx1_image_okg_20171110193848.jpg\"'" }, 
  { "name" : "cputempf", "type" : "string", "doc" : "Type inferred from '\"73.0\"'" } 
]  
}

Example Apache MiniFi Logs

2017-11-10 15:13:53,061 INFO [Provenance Maintenance Thread-3] o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers for events starting with ID 51004
2017-11-10 15:13:53,084 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.lucene.SimpleIndexManager Index Writer for provenance_repository/index-1503524885000 has been returned to Index Manager and is no longer in use. Closing Index Writer
2017-11-10 15:13:53,086 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files (6 records) into single Provenance Log File provenance_repository/50998.prov in 28 milliseconds
2017-11-10 15:13:53,087 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance Event file containing 70 records. In the past 5 minutes, 29 events have been written to the Provenance Repository, totaling 18.54 KB
2017-11-10 15:14:08,531 INFO [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@60bcd09e Successfully refreshed Peer Status; remote instance consists of 1 peers
2017-11-10 15:14:38,658 WARN [ExecuteProcess c216f845-1839-3f3c-0000-000000000000 Task] o.a.n.processors.standard.ExecuteProcess ExecuteProcess[id=c216f845-1839-3f3c-0000-000000000000] [15:14:38] src/nnvm/legacy_json_util.cc:190: Loading symbol saved by previous version v0.8.0. Attempting to upgrade...
2017-11-10 15:14:38,665 WARN [ExecuteProcess c216f845-1839-3f3c-0000-000000000000 Task] o.a.n.processors.standard.ExecuteProcess ExecuteProcess[id=c216f845-1839-3f3c-0000-000000000000] [15:14:38] src/nnvm/legacy_json_util.cc:198: Symbol successfully upgraded!
2017-11-10 15:14:38,716 WARN [ExecuteProcess c216f845-1839-3f3c-0000-000000000000 Task] o.a.n.processors.standard.ExecuteProcess ExecuteProcess[id=c216f845-1839-3f3c-0000-000000000000] /media/nvidia/96ed93f9-7c40-4999-85ba-3eb24262d0a5/mxnet/python/mxnet/module/base_module.py:65: UserWarning: Data provided by label_shapes don't match names specified by label_names ([] vs. ['softmax_label'])
2017-11-10 15:14:38,717 WARN [ExecuteProcess c216f845-1839-3f3c-0000-000000000000 Task] o.a.n.processors.standard.ExecuteProcess ExecuteProcess[id=c216f845-1839-3f3c-0000-000000000000]   warnings.warn(msg)
2017-11-10 15:14:38,965 WARN [ExecuteProcess c216f845-1839-3f3c-0000-000000000000 Task] o.a.n.processors.standard.ExecuteProcess ExecuteProcess[id=c216f845-1839-3f3c-0000-000000000000] HIGHGUI ERROR: V4L/V4L2: VIDIOC_S_CROP

Resources

Flow Files

storejetsontx1.xml

j etsontx1mx-10nov2017.xml

Cloudera Community

Community Articles

Edge Analytics with NVidia Jetson TX1 Running Apache MXNet, TensorRT, Apache MiniFi and Python Part 1 of 2

Apache MiNiFi

Apache NiFi

Tensorflow