Hello,
I am just starting to learn NiFi and I really appreciate the power of this great tool.
But I am struggling for several evenings to get what seems a simple flow to work correctly. I have reviewed several samples on how to make an API call and use pagination to retrieve difference pages. This seems to work but I get a unique FF every time. Usign MergeCOntent I was hoping I could create a combined file containing JSON array of all FF content.
It seems so simple but I am missing something. Could anyone point me in the right direction?
Logic:
1) Create FF with starting values limit (500) and start (0)
2) InvokeHTTP with URL and starting values
3) Update filename with filename prefix (object_type) and current date YYYY-MM-DD
4) Check for additional pages in Pagination group (pagination seems to work, getting multiple pages with different JSON data)
5) Iterate through InvokeHTTP with new start offset for next page
6) Using both next_page and Output output streams to ensure all pages are provided
7) MergeContent: try to merge all generate FF (by InvokeHTTP) and group by Correlation Attribute filename (should be the same during each iteration)
😎 UpdateAttribute to set final filename including hh:mm:ss
9) write single FF with merged JSON array
Problem: I still get separate FF after MergeContent.
Seems the configuration of MergeContent do not create single FF.
W
