Hi, i am trying to create a flow that forwards merged content only if there are 50 flowfiles in it over 1 minute. If there are less than 50, i want to drop the content. Is this possible without a custom processor?
Hi @Michal R, sorry i should have been more clear, I use the MergeContent processor to merge the files. I want to know if there is a way to only forward the merged content if there are 50 files in it over a certain time. So if 50 files are merged with 1 minute then forward them, if not, drop them. I need to edit my question, sorry again.
@adrian white Try to play around with attributes that are output by the merge content processor - merge.count and merge.bin.age. Depending on merge.bin.age value you can root the flow forward or to "/dev/null" equivalent.
You can use the MergeContent processor followed by a RouteOnAttribute processor to accomplish what you are looking to do.
The MergeContent processor writes an attribute named "merge.count" to the FlowFile containing all your merged source flowfiles.
- Set Min entries to "50"
- Set Max Entries to "50"
- Set Max Bin age to "1 min"
The bin age timer is trigger once the very first FlowFile is added. At the end of 1 min or 50 FlowFiles (whichever occurs first) the Bin will be merged.
Connect the "merged" relationship to a RouteOnAttribute processor that checks the "merge.count" on the merged FlowFiles to verify that they contain 50 entries.
"small" will become a new relationship to the RouteOnAttribute processor.
- Auto-terminate the "small" relationship so that any merged FlowFile with fewer then 50 entries is deleted.
- Route the "unmatched" relationship on down the rest of your dataflow.