In preparation for my talk at the Philadelphia Open Source Conference(https://phillyopensource.splashthat.com/), Apache Deep Learning 201, I wanted to have some good images for running various Apache MXNet GluonCV Deep Learning Algorithms for Computer Vision. See: https://gluon-cv.mxnet.io/
Using Apache open source tools - Apache NiFi 1.8 and Apache MXNet 1.3 with GluonCV I can easily ingest live traffic camera images and run Object Detection, Semantic Segmentation and Instance Segmentation.
Code: https://github.com/tspannhw/ApacheDeepLearning201
It's so easy, I am running multiple on the data to see which gives me the results I like. I am like YOLO which is a one of my old favorites.
- yolonifitraffic.py
- demo_mask_rcnn_nyc.py
- demo_deeplab_nyc.py
So we can find the cars and people in these webcams. Use cases can be around traffic optimization, public safety and advertisement optimization. Due to licensing, I thought it better not to show Traffic Camera data here.
To industrialize and scale out this process from a single Data Scientist to a national ingestion system, we use the power of Apache NiFi to ingest, process and control flows. I am using the latest Apache NiFi 1.8.
Apache NiFi Flow to Ingest and Process Traffic Camera Data
First we have a list of URLs that I want to process, this can be sourced and stored anywhere. For ease of use with a static set I am using GenerateFlowFile. I have a JSON file of URLs that I split and parse to call various Computer Vision Python scripts (DeepLab3, MaskRCNN, YOLO and others). YOLO seems to be the most useful so far. I am grabbing the results, some system metrics, metadata and the deep learning analytics generated by Apache MXNet.
I split the flow into two portions. One builds GluonCV result data from YOLO and the other creates a file from TensorFlow results done on the fly.
Here is a list of my webcam URLs. There's millions of them out there.
If your data is tabular, then you need a schema for fast record processing.
An Example Dataset Returned from GLUONCV - YOLO Python 3.6 Script
I turn JSON data into HDFS Writeable AVRO Data and Can Run Live SQL on It
One Output Source Code Be a Joint Slack Group
Object Detection: GluonCV YOLO v3 and Apache NiFi
This can be OpenCV, a static photo or from a URL.
Object Detection: Faster RCNN with GluonCV
Faster RCNN model trained on Pascal VOC dataset with ResNet-50 backbone
net = gcv.model_zoo.get_model(faster_rcnn_resnet50_v1b_voc, pretrained=True)
https://gluon-cv.mxnet.io/api/model_zoo.html
Instance Segmentation: Mask RCNN with GluonCV
Mask RCNN model trained on COCO dataset with ResNet-50 backbone
net = model_zoo.get_model('mask_rcnn_resnet50_v1b_coco', pretrained=True)
https://gluon-cv.mxnet.io/build/examples_instance/demo_mask_rcnn.html
https://github.com/matterport/Mask_RCNN
https://arxiv.org/abs/1703.06870
Photo by Ryoji Iwata on Unsplash
There's a lot of people crossing the street!
Semantic Segmentation: DeepLabV3 with GluonCV
GluonCV DeepLabV3 model on ADE20K dataset
model = gluoncv.model_zoo.get_model('deeplab_resnet101_ade', pretrained=True)
run1.sh demo_deeplab_webcam.py
This runs pretty slow on a machine with no GPU.
https://www.cityscapes-dataset.com/
http://groups.csail.mit.edu/vision/datasets/ADE20K/
https://arxiv.org/abs/1706.05587
https://gluon-cv.mxnet.io/build/examples_segmentation/demo_deeplab.html
That is the best picture of me ever!
Semantic Segmentation: Fully Convolutional Networks
GluonCV FCN model on PASCAL VOC dataset
model = gluoncv.model_zoo.get_model(‘fcn_resnet101_voc ', pretrained=True)
run1.sh demo_fcn_webcam.py
https://gluon-cv.mxnet.io/build/examples_segmentation/demo_fcn.html
https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf
It found me.
For NYC Dot and PennDot camera usage, you have to sign a developer agreement for a feed!
See:
http://www.nyc.gov/html/dot/html/home/home.shtml
https://www.penndot.gov/Pages/default.aspx
