Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (2)
Super Guru

Integrating Keras (TensorFlow) YOLOv3 Into Apache NiFi Workflows

For this article I wanted to try the new YOLOv3 that's running in Keras.

76432-cat.jpg

76433-cup.jpg

76434-yolokeraslivestream.png

Out of the box with video streaming, pretty cool:

git clone https://github.com/qqwweee/keras-yolo3 

wget https://pjreddie.com/media/files/yolov3.weights 

python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5 

python yolo.py

See:

https://github.com/qqwweee/keras-yolo3

My article on Darknet original YOLO v3 is here: https://community.hortonworks.com/articles/191259/integrating-darknet-yolov3-into-apache-nifi-workfl...

Example JSON Output

{
  "boxes" : "Found 8 boxes for img",
  "class7" : "diningtable",
  "score7" : "0.7484486",
  "left7" : "636",
  "right7" : "1096",
  "top7" : "210",
  "bottom7" : "693",
  "class6" : "sofa",
  "score6" : "0.31372178",
  "left6" : "1114",
  "right6" : "1276",
  "top6" : "172",
  "bottom6" : "381",
  "class5" : "chair",
  "score5" : "0.3455438",
  "left5" : "990",
  "right5" : "1048",
  "top5" : "183",
  "bottom5" : "246",
  "class4" : "chair",
  "score4" : "0.34554565",
  "left4" : "858",
  "right4" : "1000",
  "top4" : "186",
  "bottom4" : "244",
  "class3" : "chair",
  "score3" : "0.87056005",
  "left3" : "1114",
  "right3" : "1276",
  "top3" : "172",
  "bottom3" : "381",
  "class2" : "chair",
  "score2" : "0.9683409",
  "left2" : "958",
  "right2" : "1151",
  "top2" : "203",
  "bottom2" : "482",
  "class1" : "cup",
  "score1" : "0.49115792",
  "left1" : "691",
  "right1" : "770",
  "top1" : "84",
  "bottom1" : "229",
  "class0" : "person",
  "score0" : "0.9980049",
  "left0" : "187",
  "right0" : "709",
  "top0" : "2",
  "bottom0" : "720",
  "host" : "HW13125.local.fios-router.home",
  "end" : "1527443620.644303",
  "te" : "4.380625247955322",
  "battery" : 100,
  "systemtime" : "05/27/2018 13:53:40",
  "cpu" : 19.7,
  "diskusage" : "140607.7 MB",
  "memory" : 65.8,
  "yoloid" : "20180527175341_284c6051-a45d-4757-977a-fe1dd76f295b"
}

I created a YOLO ID so that the JSON, the ID and the image would have that ID for keeping them in sync as we send them across various distributed networks into a cluster for storage. I also have some other metadata I thought would be helpful such as date time, run time, battery available, disk usage, cpu and memory. Those are quick and easy to grab with Python and could be useful. This is running on a Mac laptop. This could be ported to the NVIDIA Jetson TX1. It may work on the RPI3 with Movidius, but I think it may be a touch slow. Even on a Mac with no GPU and some stuff running I am getting an image every 2-3 seconds produced. It's very choppy, I would like to try this on an UBuntu workstation with a few NVidia high end GPUs and TensorFlow compiled for GPU.

NiFi Flow for Ingest of JSON and Images

76435-yolokeraslocalflowboth.png

NiFi Server to process and store data in Hive and HBase

76440-yolokerasserverreceiveflow.png

76445-yolokerasprocessing.png

Read JSON from the Logs Directory Written by Python 3

76437-yolokerasreadjson.png

Ingest Images from Images Directory Written by YOLO v3

76438-yolokerasreadimages.png

TensorFlow Analysis of Captured Image

76439-yolokerasimageanalysis.png

You can watch the data arrive

76441-receivingdata.png

To Write Data to HBase is Easy (Just create an HBase table with a column family)

76442-yolokerashbaserecord.png


Apache NiFi generates my Hive table

76443-kerasyolohiveddl.png

Once we have a generate table it is populated with our data and we can query it in Apache Zeppelin (or any JDBC/ODBC Tool)

76444-yolokeraszepp.png

Using InferAvroSchema we had a schema created, we store it in Hortonworks Schema Registry for use.

76436-yolokerasschema.png


Sample of Data Stored in HBase

 20180527175609_579164b3-031c-4993-bec1-50ff4cf58d1c       column=yolo:boxes, timestamp=1527456044351, value=Found 9 boxes for img
 20180527175609_579164b3-031c-4993-bec1-50ff4cf58d1c       column=yolo:class0, timestamp=1527456044351, value=person
 20180527175609_579164b3-031c-4993-bec1-50ff4cf58d1c       column=yolo:class1, timestamp=1527456044351, value=chair
 20180527175609_579164b3-031c-4993-bec1-50ff4cf58d1c       column=yolo:class2, timestamp=1527456044351, value=chair
 20180527175609_579164b3-031c-4993-bec1-50ff4cf58d1c       column=yolo:class3, timestamp=1527456044351, value=chair
 20180527175609_579164b3-031c-4993-bec1-50ff4cf58d1c       column=yolo:class4, timestamp=1527456044351, value=chair
 20180527175609_579164b3-031c-4993-bec1-50ff4cf58d1c       column=yolo:class5, timestamp=1527456044351, value=chair
 20180527175609_579164b3-031c-4993-bec1-50ff4cf58d1c       column=yolo:class6, timestamp=1527456044351, value=chair
 20180527175609_579164b3-031c-4993-bec1-50ff4cf58d1c       column=yolo:class7, timestamp=1527456044351, value=sofa
 20180527175609_579164b3-031c-4993-bec1-50ff4cf58d1c       column=yolo:class8, timestamp=1527456044351, value=diningtable


Flow (Local, Server) and Zeppelin Notebook:

yolo-keras-json-server-save.xml

keras-tensorflow-yolo-v3-osx.xml

yolo-copy.json


Forked Python Code For Saving JSON and Images

https://github.com/tspannhw/yolo3-keras-tensorflow

Coming Soon:

Live Recording of YOLO v3 with Keras/TensorFlow recording of the capture stream.


1,691 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 07:19 AM
Updated by:
 
Contributors
Top Kudoed Authors