Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Super Guru

Capture Images from PicSum.com Free Images

80476-captureimagesflow.png

Process All the Images via TensorFlow Processor, SSD Predict via MMS and SqueezeNet v1.1 via MMS

80477-mmslocalflow.png


Apache Zeppelin SQL Against tblsqueeze11

80458-mms2-zep-squeeze11select.png

Example Output from Squeeze v1.1

80459-mms2-squeezedata.png

Storing Generic Data in HDFS via Schema

80460-mms2-puthdfs.png

Example SSD Data JSON

80461-mms2-ssdataexample.png

High Level Flow From Server

80462-mms2-overviewserverflow.png

Apache NiFi Server Flows to Store

80463-mms2-serverprocessing-flow1.png

80464-mms2-serverprocessing-flow2.png

Convert to Apache ORC

80465-mms2-convertorc.png

Extract Attributes

80466-mms2-inferavro.png

Convert JSON Arrays to Other

80467-mms2-splitjson1.png

80468-mms2-splitjson2.png

Example Data Derived From TensorFlow Processor

80469-mms2-exampletensorflowdata.png

Schemas in Schema Registry

80470-mms2-tensorflowschema.png

80471-mms2-squeeze11schema.png

Create Table in Zeppelin

80472-mms2-zep-createtbltensorflow.png

Query Table in Zeppelin

80473-mms2-zep-tbltensorflow.png


Python Libraries

git clone https://github.com/awslabs/mxnet-model-server.git


pip install opencv-python  -U
pip install scikit-learn -U
pip install easydict -U
pip install scikit-image -U
pip install numpy -U
pip install mxnet -U

pip3.6 install opencv-python -U
pip3.6 install scikit-learn -U
pip3.6 install easydict -U
pip3.6 install scikit-image -U
pip3.6 install numpy -U
pip3.6 install mxnet -U

Example Runs - Squeeze v1.1

mxnet-model-server --models squeezenet=squeezenet_v1.1.model --service mms/model_service/mxnet_vision_service.py --port 9999

[INFO 2018-07-10 16:50:26,840 PID:7730 /usr/local/lib/python3.6/site-packages/mms/request_handler/flask_handler.py:jsonify:159] Jsonifying the response: {'prediction': [[{'probability': 0.3365139067173004, 'class': 'n03710193 mailbox, letter box'}, {'probability': 0.1522996574640274, 'class': 'n03764736 milk can'}, {'probability': 0.08760709315538406, 'class': 'n03000134 chainlink fence'}, {'probability': 0.08103135228157043, 'class': 'n02747177 ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin'}, {'probability': 0.04956872761249542, 'class': 'n02795169 barrel, cask'}]]}
[INFO 2018-07-10 16:50:26,842 PID:7730 /usr/local/lib/python3.6/site-packages/werkzeug/_internal.py:_log:88] 127.0.0.1 - - [10/Jul/2018 16:50:26] "POST /squeezenet/predict HTTP/1.1" 200 -
[INFO 2018-07-10 16:50:46,904 PID:7730 /usr/local/lib/python3.6/site-packages/mms/serving_frontend.py:predict_callback:467] Request input: data should be image with jpeg format.
[INFO 2018-07-10 16:50:46,960 PID:7730 /usr/local/lib/python3.6/site-packages/mms/request_handler/flask_handler.py:get_file_data:137] Getting file data from request.
[INFO 2018-07-10 16:50:47,020 PID:7730 /usr/local/lib/python3.6/site-packages/mms/serving_frontend.py:predict_callback:510] Response is text.
[INFO 2018-07-10 16:50:47,020 PID:7730 /usr/local/lib/python3.6/site-packages/mms/request_handler/flask_handler.py:jsonify:159] Jsonifying the response: {'prediction': [[{'probability': 0.1060439869761467, 'class': 'n02536864 coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch'}, {'probability': 0.06582894921302795, 'class': 'n01930112 nematode, nematode worm, roundworm'}, {'probability': 0.05008145794272423, 'class': 'n01751748 sea snake'}, {'probability': 0.03847070038318634, 'class': 'n01737021 water snake'}, {'probability': 0.03614763543009758, 'class': 'n09229709 bubble'}]]}
[INFO 2018-07-10 16:50:47,021 PID:7730 /usr/local/lib/python3.6/site-packages/werkzeug/_internal.py:_log:88] 127.0.0.1 - - [10/Jul/2018 16:50:47] "POST /squeezenet/predict HTTP/1.1" 200 -


mxnet-model-server --models SSD=resnet50_ssd_model.model --service ssd_service.py --port 9998


Apache MXNet Model Server Model Zoo

https://github.com/awslabs/mxnet-model-server/blob/master/docs/model_zoo.md


Connect to MMS

/opt/demo/curl.sh
curl -X POST http://127.0.0.1:9998/SSD/predict -F "data=@$1" 2>/dev/null

/opt/demo/curl2.sh
curl -X POST http://127.0.0.1:9999/squeezenet/predict -F "data=@$1" 2>/dev/null


Flows

Reference

388 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 07:01 AM
Updated by:
 
Contributors
Top Kudoed Authors