I created a YOLO ID so that the JSON, the ID and the image would have that ID for keeping them in sync as we send them across various distributed networks into a cluster for storage. I also have some other metadata I thought would be helpful such as date time, run time, battery available, disk usage, cpu and memory. Those are quick and easy to grab with Python and could be useful. This is running on a Mac laptop. This could be ported to the NVIDIA Jetson TX1. It may work on the RPI3 with Movidius, but I think it may be a touch slow. Even on a Mac with no GPU and some stuff running I am getting an image every 2-3 seconds produced. It's very choppy, I would like to try this on an UBuntu workstation with a few NVidia high end GPUs and TensorFlow compiled for GPU.
NiFi Flow for Ingest of JSON and Images
NiFi Server to process and store data in Hive and HBase
Read JSON from the Logs Directory Written by Python 3
Ingest Images from Images Directory Written by YOLO v3
TensorFlow Analysis of Captured Image
You can watch the data arrive
To Write Data to HBase is Easy (Just create an HBase table with a column family)
Apache NiFi generates my Hive table
Once we have a generate table it is populated with our data and we can query it in Apache Zeppelin (or any JDBC/ODBC Tool)
Using InferAvroSchema we had a schema created, we store it in Hortonworks Schema Registry for use.