Member since
02-19-2018
2
Posts
0
Kudos Received
0
Solutions
02-20-2018
10:41 PM
Thanks for your replies! I have small json objects coming in the MergeRecord processor and I'm merging them in a csv file The upper limit is 1,000,000 per hour, processing is very fast As a workaround I have it setup with min bin size to 1,000,000 and max bin age 1h but this means I have to start it exactly on the hour plus it drifts as time goes by. Because the processing time is not fixed (different loads for different hours) even if I set the bin age to something like 59min it will still drift up or down
... View more
02-19-2018
11:27 PM
Hi guys, Working on a 3 machine NiFi cluster and trying to get MergeRecord to merge all the items in its queue every hour. I set CRON scheduling to 0 0 * * * ? and all sorts of combinations of values for bin size and age but nothing worked, it either just gets a few items at a time or none at all. If I set it to timer driven all the bin settings work exactly as I expect them to. I think there is some sort of connection between having it run once a hour and the max bin age, seems to be some sort of a deadlock condition but I don't understand exactly what's going on. Thanks a lot!
... View more
Labels:
- Labels:
-
Apache NiFi