Member since
11-22-2018
7
Posts
0
Kudos Received
0
Solutions
11-23-2018
10:26 PM
@Shu I am sorry that i didnt mention this earlier, i actually use correlation attribute "filename" and i would be having 500 filenames/second. Thanks for the links.
... View more
01-16-2019
02:23 PM
@Jose Paul - A bin would be eligible for merge with only 1 FlowFile in it since you set minEntries to 1. - When the Processor get scheduled to execute (based on configured run schedule and scheduling strategy), It will look at one of possible many incoming connections and look at only the queued FlowFile at that exact moment in time. It will then bin those FlowFiles based on configuration. So it multiple FlowFiles happen to exist in that connection with sam filename attribute value, they will be placed in same bin. At completion of of placing those FlowFiles in bins, the bins are evaluated if they are eligible to be merged. In your case since minEntries is 1 all bins with 1 or more FlowFiles would be merged. - If you run schedule is set to run as fast as possible (Timer Driven with run schedule of 0 sec), it may be reading the inbound connection so fast that it only contains 1 or just a few FlowFiles per execution. - The other scenario is an inbound connection with over 500 queued FlowFiles at time of execution. If we assume there are more than 500 FlowFiles with unique values assigned to the filename attribute, each would end up be placed in new bin (correlation attribute config). As soon as bin 500 has a FlowFile assigned to it and MergeContent tries to bin unique filename number 501, it has no available bins left so it forces the merging of the oldest bin to free a bin. - Thank you, Matt
... View more