Posting Images with Apache NiFi 1.7 and a Custom Processor

[#1]

Posting Images with Apache NiFi 1.7 and a Custom Processor

I have been using a shell script for this since Apache NiFi did not have a good way to natively post an image to HTTP servers su such as the model server for Apache MXNet.

So I wrote a quick and dirty processor that posts an image there and gathers the headers, result body, status text and status code and returns them to you as attributes.

In this example I am download images from picsum.photos free photo service.

92696-postimagenififlow.png

To use this new processor, download to your lib directory and restart Apache NiFi, then you can add the PostImageProcessor.

92697-addpostimageprocessor.png

Eclipse For Building My Processor

92698-nifihttppost.png

Configure the Post Image Processor with your URL, fieldname, imagename and image type.

92699-configurepostimage.png

MXNet Model Server Results

92700-callmmsfromnifi.png

The Attribute Results From the Data

92701-postimageprovenance.png

Results

92702-postimageprocessorprovenance0.png

Example Results

post.header
{Server=[Werkzeug/0.14.1 Python/3.6.6], Access-Control-Allow-Origin=[*], Content-Length=[396], Date=[Fri, 05 Oct 2018 17:47:22 GMT], Content-Type=[application/json]}
post.results
{"prediction":[[{"probability":0.24173378944396973,"class":"n02281406 sulphur butterfly, sulfur butterfly"},{"probability":0.19173663854599,"class":"n02190166 fly"},{"probability":0.052654966711997986,"class":"n02280649 cabbage butterfly"},{"probability":0.05147545784711838,"class":"n03485794 handkerchief, hankie, hanky, hankey"},{"probability":0.048753462731838226,"class":"n02834397 bib"}]]}
post.status
OK
post.statuscode
200 

Results from HTTP Posting an Image to MXNet Model Server

[INFO 2018-10-05 13:47:22,217 PID:88561 /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mms/serving_frontend.py:predict_callback:467] Request input: data should be image with jpeg format.
[INFO 2018-10-05 13:47:22,218 PID:88561 /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mms/request_handler/flask_handler.py:get_file_data:137] Getting file data from request.
[INFO 2018-10-05 13:47:22,262 PID:88561 /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mms/serving_frontend.py:predict_callback:510] Response is text.
[INFO 2018-10-05 13:47:22,262 PID:88561 /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mms/request_handler/flask_handler.py:jsonify:159] Jsonifying the response: {'prediction': [[{'probability': 0.24173378944396973, 'class': 'n02281406 sulphur butterfly, sulfur butterfly'}, {'probability': 0.19173663854599, 'class': 'n02190166 fly'}, {'probability': 0.052654966711997986, 'class': 'n02280649 cabbage butterfly'}, {'probability': 0.05147545784711838, 'class': 'n03485794 handkerchief, hankie, hanky, hankey'}, {'probability': 0.048753462731838226, 'class': 'n02834397 bib'}]]}
[INFO 2018-10-05 13:47:22,263 PID:88561 /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/werkzeug/_internal.py:_log:88] 127.0.0.1 - - [05/Oct/2018 13:47:22] "POST /squeezenet/predict HTTP/1.1" 200 -

Example HTTP Server

https://github.com/awslabs/mxnet-model-server

Source Code For Processor

https://github.com/tspannhw/nifi-postimage-processor

Pre-Built NAR To Install

https://github.com/tspannhw/nifi-postimage-processor/releases/tag/1.0