InvokeHTTP POST with parameter

[#1]

It seems this question gets asked a lot in this forum, but there aren't any good responses.  Hopefully somebody can help me out here.

I am attempting to use the InvokeHTTP processor to POST a query to an REST endpoint.  I currently use the InvokeHTTP processor to access data on this data service using the GET method, and that always works fine.  However, there is a specific service endpoint that only accepts POST requests.  I've verified this service is functional using curl.  But when I attempt to use the NiFi InvokeHTTP processor, I get no response.  Hopefully somebody here knows what's going on.  Below are some screen shots of my processor configuration.

If I understand the documentation correctly, I need to specify in the "Attributes to Send" field the flowfile attributes I want to send as parameters.  Since this processor is the first processor in the flow, it does't have any attributes yet, so I add a dynamic attribute named "data" (shown at the bottom).  Then I reference that attribute in the "Attributes to Send" field.  Seems correct per the documentation.  But it don't work.  In fact I get no response at all, not even a 400 series response.

What am I missing here?  This seems like it should be really simple, but yet is bedeviling me.

[#2]

Hi,

Few pointers that might help:

- I found that when the "Always Output Response" is set to False you are not going to get a response flowfile specially when there are errors, so make sure to set it to True to get response all the time.

- My understanding is that you are trying to send Query Parameters with param name "data". If that is correct, then what you specified is not going to work because dynamic property and "Attributes to Send" is for API header info and not query param. To send the query param you need to include it as part of the "Remote Url" expression:

#{ADSB_url}/?data=${data}

I suggest you first try to the exact url in the Remote url value to make sure its working as expected , then work your way back to using Parameter Context and Attributes.

Hope that helps.

[#3]

Thanks SAMSAL. 

To clarify, I am not sending query parameters.  I am attempting to perform a POST so I need to specify my parameter as an attribute.

I changed the "Always Output Response" to True which seems like a good idea but unfortunately that didn't change anything.  Which is really weird, because I should be betting some sort of response.  

Is there any way I can see the HTTP payload that NiFi submits?  I've combed through the log files and I can't find any references to InvokeHTTP traffic.  If I could review the payload then I can figure out what is messed up.  Any ideas about how to view the HTTP traffic?

[#4]

hmmm, Are you sure the inbokehttp is getting triggered? how is it  getting triggered ? did you try to run it manually using run-once ? Also can you show how the api is getting call using curl (you can hide the full url)?

[#5]

Yes, I am certain.  Let me explain why and share what I've done so far.

1. When I attempt to monitor the network traffic, I can't see the outgoing payload from my browser because the call is made from the server, however, I can see 401 and a 403 errors returned.  

2. I can change the REST endpoint to one that uses a GET and this works fine

3. I can POST a write operation, and that works fine

4. I used the PostHTTP processor, and this processor actually correctly returns the 401 and 403 Responses, so this confirms what I saw via the network traffic

So at this point I am certain the issue is an authentication issue.  But why?

I'm using "OAuth2 Access Token" successfully in every other instance with this API.  So I experimented with the PostHTTP processor by adding the token as an attribute to pass along.   That still didn't work.  What is the correct way to manually submit an authorization token?

Is there any way for me to log these HTTP calls from within NiFi?

[#6 · Accepted solution]

I figured this out.  The problem was that I was missing two things.

1. I need another processor BEFORE the InvokeHTTP processor to send in a flow file and trigger the InvokeHTTP processor.  Seems obvious now, but didn't dawn on me at the time. 

2. I need the sql query to be in the body of the flow file, which I accomplish by using a GenerateFlowFile processor in front of the InvokeHTTP processor, and then set the  "Custom Text" field to my SQL query.  This also satisfies the first requirement of needing an incoming flow file to trigger the InvokeHTTP processor.

It is literally that simple.  I didn't need to specify the attributes to include or anything else.  However, I did specify the Content-Type as "text/plain".