Re: NiFi MergeRecord CRON scheduling

cfluture · ‎02-20-2018

Thanks for your replies! I have small json objects coming in the MergeRecord processor and I'm merging them in a csv file The upper limit is 1,000,000 per hour, processing is very fast As a workaround I have it setup with min bin size to 1,000,000 and max bin age 1h but this means I have to start it exactly on the hour plus it drifts as time goes by. Because the processing time is not fixed (different loads for different hours) even if I set the bin age to something like 59min it will still drift up or down

cfluture · ‎02-19-2018

Hi guys, Working on a 3 machine NiFi cluster and trying to get MergeRecord to merge all the items in its queue every hour. I set CRON scheduling to 0 0 * * * ? and all sorts of combinations of values for bin size and age but nothing worked, it either just gets a few items at a time or none at all. If I set it to timer driven all the bin settings work exactly as I expect them to. I think there is some sort of connection between having it run once a hour and the max bin age, seems to be some sort of a deadlock condition but I don't understand exactly what's going on. Thanks a lot!

Online	Offline
Last Visited	‎02-20-2018 11:32 PM

Date Registered	‎02-19-2018 11:15 PM
Last Visited	‎02-20-2018 11:32 PM
Total Messages Posted	2