NiFi InvokeHTTP POST JSON

[#1]

I'm totally new to NiFi and learning as I go, but am against a hard deadline and need urgent assistance. I was tasked to create a data interface between two learning systems to send course completion data.

I'm creating a flow to send course completion data from one system to another and have a JSON formatted file (to send as HTTP body to send as a CURL post?)

This is the sample JSON data that I've created so far:
{"user_id":10042,"username":"de","course_id":"C00230","completed_on":"12-02-2020","SCORE":100}

The suggested HTTPS Curl request format is provided below.
curl -v -X POST -d "j_username=admin&j_password=admin&userID=MORRW001&crsID=C00565&dateCompleted=2020-11-16&testScore=100&courseType=a" https://lms.dev.test.net/jw/web/json/data/

How do I configure the InvokeHTTP processor to send it correctly? Do I need to reformat the JSON data so it's in the correct format for the curl format shown above?

Remember I'm just a NiFi novice, so details, including screenshots would be appreciated.

Thanks in advance!

[#2 · Accepted solution]

@GMAN  Here is my try to help you further.

flow

Look at the picture of the InvokeHTTP.

InvokeHTTP

From my experience most options can mostly be left to default values.

Your Json-body should be fine and will be send with the request.

Remarks:
- Authenticaton/Authorization: You have to try the two possibilities I know. For the second you have to add custom-property "Authorization" with the plus symbol on the right above.

- Accept: Concerns the response format not sure wether you will need this.

Maybe you can first build your request in Postman and after successfull execution then click to "code" -> "HTTP". There you can see the details needed for the request.

In the "Settings" of the InvokeHTTP you can autoterminate the "original" flowfile.
For "Retry" you can use the shown processor with default values.
All "Failure" or "No retry" go to some error-handling.
With "Response" you can go further in your flow.

Hope this helps you to come forward.

[#3 · Accepted solution]

@stevenmatisonThanks for telling me about the templates.   I downloaded these and will use as a guide.  

[#4 · Accepted solution]

@GMAN  Here is my try to help you further.

flow

Look at the picture of the InvokeHTTP.

InvokeHTTP

From my experience most options can mostly be left to default values.

Your Json-body should be fine and will be send with the request.

Remarks:
- Authenticaton/Authorization: You have to try the two possibilities I know. For the second you have to add custom-property "Authorization" with the plus symbol on the right above.

- Accept: Concerns the response format not sure wether you will need this.

Maybe you can first build your request in Postman and after successfull execution then click to "code" -> "HTTP". There you can see the details needed for the request.

In the "Settings" of the InvokeHTTP you can autoterminate the "original" flowfile.
For "Retry" you can use the shown processor with default values.
All "Failure" or "No retry" go to some error-handling.
With "Response" you can go further in your flow.

Hope this helps you to come forward.

[#5]

@justenji   Great response!

@GMAN  I have some templates that may help you get a head start:

https://github.com/steven-matison/NiFi-Templates

There are 2 InvokeHttp Examples.

[#6 · Accepted solution]

@stevenmatisonThanks for telling me about the templates.   I downloaded these and will use as a guide.  

[#7]

@stevenmatisonI used your template and tried to assign attributes using EvaluateJSONPath as show here:

This is the JSON data entering the EvaluateJSONPath processor.

{"user_id":"68","course_id":"HA044","completed_on":"12-15-2020","score":null}

However only the "crsID" and "dateCompleted" attributes are showing as attributes in the queue after this processor and I don't understand why.  

[#8]

Thats weird.. did you scroll the attributes list down (just in case its that simple)?   i cant see the full modal so not sure.

The config of each attribute looks okay so does the json.

[#9]

@stevenmatisonYou were correct as the other attributes were listed at the bottom of the Attribute Values.   I didjn't notice the scroll bar and needed to scroll down.  

Thanks again.

[#10 · Accepted solution]

Excellent news.  Can you accept the first 2 responses to close this solution?