How to create a json request for invokehttp processor from flowfile and replace the values in flowfile with response : NiFi

[#1]

Hi,

I have a flowfile json request which looks like this. Actually flowfile is a csv file which I am converting to json first so that another json request can be created for invokehttp processor. But if this csv to json conversion is not required for it then I would prefer to keep it csv only.

{
"type": "record",
"name": "Location",
"fields": [{
"name": "Company_name",
"type": ["null", "string"]
},
{
"name": "City",
"type": ["null", "string"]
},
{
"name": "Country",
"type": ["null", "string"]
},
{
"name": "Latitude",
"type": ["null", "string"]
},
{
"name": "Longitude",
"type": ["null", "string"]

}
]
}

I need to send a request to invokehttp processor which look like below - 

{
  "Code":"A-5678",
  "Latitude":"78.495603",
  "Longitude":"5.969756",
  "Continent":""
}

Latitude and longitude is to be taken from flowfile fields while Code and Continent (To be sent as blank string) are not available in flowfile and has to be created. How can I create this custom request. I tried evaluatejsonpath processor but unable to add Continent as it does not take blank value.

Once the request is received I need to replace the Latitude and Logitude values in flow file with the response received. Please let me know how to achieve it. The final output has to be csv file.

@Shu_ashu @mburgess Please help

[#2]

@vikrant_kumar24   To add new attributes to a flow file you should use UpdateAttribute.  To get attributes from json you should use EvaluateJsonPath.  Once you have all the values you want as attributes you should use AttributesToJson and then finally send the json to invokeHttp.

Your flow will look like this:

And the Queued Flowfile waiting on InvokeHttp is:

Please note I did not have your original Json, just the schema above, so I created a simple JSON of the Longitute & Latitude values you gave me.    I also did not touch InvokeHttp, its default.   Once you get this far, and want to work on handling the response from InvokeHttp let me know here, in another post, or a private message.   To see my entire template, check out my GitHub for the Invoke Http Demo 2 Template:

https://github.com/steven-dfheinz/NiFi-Templates

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

Thanks,

Steven @ DFHZ

[#3]

Hi Steven,

Thanks a lot for responding to my query. I helps a lot. I have downloaded your flow. I have few clarification questions.

1. In Updateattribute processor in your template - It is not adding any flow file attribute. So we need to add Code and Continent there right?

2. So we need to add anything in the properites of invokehttp processor? We will get response from invokeHTTP and that response needs to replace the values of longitude and latitude in original flow file. How to do that?

3. If original flow file is a csv file instead of json. Will the same flow work or do we need to convert csv to json first?

csv file is like this

Company_name, City, Country, Latitude, Longitude

Abc Corp, Newyork, USA, 98.765, 67.5656

[#4 · Accepted solution]

@vikrant_kumar24 

1.  Yes,  sorry for some reason they didnt get saved in that version of the template.  I will update.  You should see or set it like this:

2.  Yes,  you will need to enter the Remote Url, and maybe more properties.    You also need to confirm that the json content in flow file is correct.  Mine is just an example.  I do not know what service you are connect too.  That service may require username, passwords, tokens, headers, etc.  The configuration for InvokeHttp will depend on these.   If you prefer to share more details about the service you are connecting in PM we can discuss in private.

Once you have the response from InvokeHttp,  if the response is json, you will again do EvaluateJson, grab the values you want for the new Long/Lat, and replace or overwrite the existing attributes and continue downstream with your flow.   

3.  Converting CSV to JSON is pretty typical.  The CSV->JSON Method you are a working on is just fine.  You should be able to link the flowfile from that, to EvaluateJson in the flow.    My example is just a demo to show you the concepts of EvaluateJson, UpdateAttribute, and AttributesToJson.