Member since
11-22-2018
7
Posts
0
Kudos Received
0
Solutions
11-28-2018
12:37 PM
@Matt Clarke: Thanks for the response. Can you please clarify if i use Bin Packing algorithm correlation attribute is filename,minEntries 1 and no MaxBinAge, number of MaxBins: 500 then when will be the merging happens?
... View more
11-23-2018
10:26 PM
@Shu I am sorry that i didnt mention this earlier, i actually use correlation attribute "filename" and i would be having 500 filenames/second. Thanks for the links.
... View more
11-23-2018
06:40 PM
@ShuWow.Thats a good idea to make the processor wait, thank you. I am expecting 500 flowfiles/second tops, thats is why i have kept 500 bins. If i am setting max bin age as 10 seconds,it would cost me 5000 bins right?
... View more
11-23-2018
07:03 AM
@Shu: Thanks for the response. I want to wait my mergecontent processor for sometime before it merges into a single flow file when using Bin packing algorithm (Now it seems like files are merged in less than a second) . Is there a way i can do that? I use RouteOnAttribute processor to filter so that part is okay for me. What is the exact criteria for merging in Bin Packing algorithm (Min Entry: 1, max entry: 1000, Max Bin Age 10 seconds,number of bins 500, yet i have seen files (splits less than 5) being merged under one second)
... View more
11-22-2018
02:39 PM
@sally sally @Matt Clarke Is there a way that i can specify a bin should wait for any minimum amount of time at least (min Bin Age). I use a split processor to split incoming flow files , enrich each of the split and finally merges them back to the original flow file. My process of enriching might be delayed so i want to wait till all the splits comes together, i cant use Defragment strategy as i may not have all the splits (i want to reject some splits based on some criteria). Can you please help here?
... View more
11-22-2018
01:18 PM
MergeContentProcessor - Nifi: Is there a way that i can specify a bin should wait for any minimum amount of time at least (min Bin Age). I use a split processor to split incoming flow files , enrich each of the split and finally merges them back to the original flow file. My process of enriching might be delayed so i want to wait till all the splits comes together, i cant use Defragment strategy as i may not have all the splits (i want to reject some splits based on some criteria). Can you please help here?
... View more
Labels:
- Labels:
-
Apache NiFi