Integrating Apache MXNet Model Server with Apache NiFi - Updated

[#1]

Capture Images from PicSum.com Free Images

80476-captureimagesflow.png

Process All the Images via TensorFlow Processor, SSD Predict via MMS and SqueezeNet v1.1 via MMS

80477-mmslocalflow.png

Apache Zeppelin SQL Against tblsqueeze11

80458-mms2-zep-squeeze11select.png

Example Output from Squeeze v1.1

80459-mms2-squeezedata.png

Storing Generic Data in HDFS via Schema

80460-mms2-puthdfs.png

Example SSD Data JSON

80461-mms2-ssdataexample.png

High Level Flow From Server

80462-mms2-overviewserverflow.png

Apache NiFi Server Flows to Store

80463-mms2-serverprocessing-flow1.png

80464-mms2-serverprocessing-flow2.png

Convert to Apache ORC

80465-mms2-convertorc.png

Extract Attributes

80466-mms2-inferavro.png

Convert JSON Arrays to Other

80467-mms2-splitjson1.png

80468-mms2-splitjson2.png

Example Data Derived From TensorFlow Processor

80469-mms2-exampletensorflowdata.png

Schemas in Schema Registry

80470-mms2-tensorflowschema.png

80471-mms2-squeeze11schema.png

Create Table in Zeppelin

80472-mms2-zep-createtbltensorflow.png

Query Table in Zeppelin

80473-mms2-zep-tbltensorflow.png

Python Libraries

git clone https://github.com/awslabs/mxnet-model-server.git


pip install opencv-python  -U
pip install scikit-learn -U
pip install easydict -U
pip install scikit-image -U
pip install numpy -U
pip install mxnet -U

pip3.6 install opencv-python -U
pip3.6 install scikit-learn -U
pip3.6 install easydict -U
pip3.6 install scikit-image -U
pip3.6 install numpy -U
pip3.6 install mxnet -U

Example Runs - Squeeze v1.1

mxnet-model-server --models squeezenet=squeezenet_v1.1.model --service mms/model_service/mxnet_vision_service.py --port 9999

[INFO 2018-07-10 16:50:26,840 PID:7730 /usr/local/lib/python3.6/site-packages/mms/request_handler/flask_handler.py:jsonify:159] Jsonifying the response: {'prediction': [[{'probability': 0.3365139067173004, 'class': 'n03710193 mailbox, letter box'}, {'probability': 0.1522996574640274, 'class': 'n03764736 milk can'}, {'probability': 0.08760709315538406, 'class': 'n03000134 chainlink fence'}, {'probability': 0.08103135228157043, 'class': 'n02747177 ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin'}, {'probability': 0.04956872761249542, 'class': 'n02795169 barrel, cask'}]]}
[INFO 2018-07-10 16:50:26,842 PID:7730 /usr/local/lib/python3.6/site-packages/werkzeug/_internal.py:_log:88] 127.0.0.1 - - [10/Jul/2018 16:50:26] "POST /squeezenet/predict HTTP/1.1" 200 -
[INFO 2018-07-10 16:50:46,904 PID:7730 /usr/local/lib/python3.6/site-packages/mms/serving_frontend.py:predict_callback:467] Request input: data should be image with jpeg format.
[INFO 2018-07-10 16:50:46,960 PID:7730 /usr/local/lib/python3.6/site-packages/mms/request_handler/flask_handler.py:get_file_data:137] Getting file data from request.
[INFO 2018-07-10 16:50:47,020 PID:7730 /usr/local/lib/python3.6/site-packages/mms/serving_frontend.py:predict_callback:510] Response is text.
[INFO 2018-07-10 16:50:47,020 PID:7730 /usr/local/lib/python3.6/site-packages/mms/request_handler/flask_handler.py:jsonify:159] Jsonifying the response: {'prediction': [[{'probability': 0.1060439869761467, 'class': 'n02536864 coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch'}, {'probability': 0.06582894921302795, 'class': 'n01930112 nematode, nematode worm, roundworm'}, {'probability': 0.05008145794272423, 'class': 'n01751748 sea snake'}, {'probability': 0.03847070038318634, 'class': 'n01737021 water snake'}, {'probability': 0.03614763543009758, 'class': 'n09229709 bubble'}]]}
[INFO 2018-07-10 16:50:47,021 PID:7730 /usr/local/lib/python3.6/site-packages/werkzeug/_internal.py:_log:88] 127.0.0.1 - - [10/Jul/2018 16:50:47] "POST /squeezenet/predict HTTP/1.1" 200 -


mxnet-model-server --models SSD=resnet50_ssd_model.model --service ssd_service.py --port 9998

Apache MXNet Model Server Model Zoo

https://github.com/awslabs/mxnet-model-server/blob/master/docs/model_zoo.md

Connect to MMS

/opt/demo/curl.sh
curl -X POST http://127.0.0.1:9998/SSD/predict -F "data=@$1" 2>/dev/null

/opt/demo/curl2.sh
curl -X POST http://127.0.0.1:9999/squeezenet/predict -F "data=@$1" 2>/dev/null

Flows

• mxnetserverlocal.xml
• mxnetmodelserver.xml

Reference

• https://community.hortonworks.com/articles/155435/using-the-new-mxnet-model-server.html
• https://community.hortonworks.com/articles/177232/apache-deep-learning-101-processing-apache-mxnet-m...
• https://mxnet.incubator.apache.org/model_zoo/
• https://medium.com/apache-mxnet/mxnet-1-2-adds-built-in-support-for-onnx-e2c7450ffc28
• https://mxnet.incubator.apache.org/api/python/gluon/model_zoo.html
• https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challen...
• https://github.com/onnx/models
• https://github.com/awslabs/mxnet-model-server/blob/master/docs/model_zoo.md#lstm-ptb
• https://github.com/awslabs/mxnet-model-server/blob/master/docs/model_zoo.md#arcface-resnet100_onnx