question Re: Can you retrieve specified attributes from a url with InvokeHTTP? in Archives of Support Questions (Read Only)

Can you retrieve specified attributes from a url with InvokeHTTP?

r_young — Wed, 01 Mar 2017 01:40:18 GMT

Hi, currently i am using InvokeHTTP to retrieve a JSON file from a weather website. Then i use EvaluateJsonPath to take the attributes i need from the json file. Is it possible to just recieve the attributes from a url instead of receiving the whole json file? Thank you

Re: Can you retrieve specified attributes from a url with InvokeHTTP?

mburgess — Wed, 01 Mar 2017 02:04:18 GMT

What do you mean by receiving attributes from a URL? Do you mean query parameters? Or are you asking if there is a way to combine the fetching of the document via the URL and the extraction of JSON fields from the response? If the latter, there isn't a way to combine those currently; NiFi is a highly modular system so fetching is really a different operation than extractions, hence the two separate processors.

What do you see as the issue in receiving the whole JSON response? The third-party libraries I'm familiar with still retrieve the whole HTTP response whether they expose it via an input stream or a big string or whatever.

After the initial fetch/response, if you don't touch the content after that, there won't be an operational impact with the respect to the size of the JSON response. If you no longer need the content, you can replace it with something else (or nothing) using ReplaceText.

One concept that is (hopefully) becoming more popular with web services is support for GraphQL. Using that paradigm and query language, you could ask for just the fields/structures you want, and you get the response you expect.

Re: Can you retrieve specified attributes from a url with InvokeHTTP?

r_young — Wed, 01 Mar 2017 03:44:26 GMT

Hi @Matt Burgess yes its the latter. No issue with receiving the whole JSON, i was just trying to minimize the response if possible.