Support Questions
Find answers, ask questions, and share your expertise

page through results - InvokeHTML process through NIFI

New Contributor

Hi, I have created a flow using the Invokehtml and by default, the endpoints mentioned in the invoke function will return 300 activity items at a time.  It also has moreResult attribute is if it's true, this means that more results are available. How do I continue to call the endpoint until the moreResult attribute returns false, which means that there are no more results available. ?

4 REPLIES 4

@ajay_mamgain200  After the first invokeHtml you should do a check for the moreResults attribute to exist.  If it exists, route that FlowFile back to invokeHttml to run again.  If does not exist or false, go to next step in your flow.

 

Hope this helps..

New Contributor

Thank you for the quick response @stevenmatison  I have done the exact same however I'm not able to get the new data it's just throwing the same data again and again. image.pngimage.pngimage.png

@ajay_mamgain200   Try to go back to the source system, and see if there is something you need to pass on the additional nth calls to indicate you want the "more results" not the initial results.  Sometimes this is another flag, or a range to identify a start/stop point.    If the additional call is quite a bit different, do not loop back to the original invokeHttp, use a separate invokeHttp and loop to this one until done; unless you can make a single InvokeHttp dynamically handle first and additional nth calls.   

 

I would test these calls in postman before attempting in NiFi, so that you know for sure what the first call is, and what 2nd, 3rd, nth is.

New Contributor

Thank you for the response @stevenmatison I can confirm that we need to make the same API request which I have used in the invokehtml initially. I have done the testing in PowerBI because I can append the data in the list and i was able to get the data. Do you know if we can generate a list nifi and append the data ? It will be difficult to do that in Postman but it does work in Python as well.

 

GeneratedList =
List.Generate(
()=>[i=0, res = FnProcessOnePage(accessTokenParamStr, pagingTokenParamStr)],
each [i]<iterationsNum and [res][Data]<>null,
each [i=[i]+1, res = FnProcessOnePage([res][Access],[res][Next])],
each [res][Data])