cfluture
New Contributor

Re: NiFi MergeRecord CRON scheduling

by New Contributor in Support Questions
‎02-20-2018 10:41 PM
‎02-20-2018 10:41 PM
Thanks for your replies! I have small json objects coming in the MergeRecord processor and I'm merging them in a csv file The upper limit is 1,000,000 per hour, processing is very fast As a workaround I have it setup with min bin size to 1,000,000 and max bin age 1h but this means I have to start it exactly on the hour plus it drifts as time goes by. Because the processing time is not fixed (different loads for different hours) even if I set the bin age to something like 59min it will still drift up or down ... View more

NiFi MergeRecord CRON scheduling

by New Contributor in Support Questions
‎02-19-2018 11:27 PM
‎02-19-2018 11:27 PM
Hi guys, Working on a 3 machine NiFi cluster and trying to get MergeRecord to merge all the items in its queue every hour. I set CRON scheduling to 0 0 * * * ? and all sorts of combinations of values for bin size and age but nothing worked, it either just gets a few items at a time or none at all. If I set it to timer driven all the bin settings work exactly as I expect them to. I think there is some sort of connection between having it run once a hour and the max bin age, seems to be some sort of a deadlock condition but I don't understand exactly what's going on. Thanks a lot! ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎02-20-2018 11:32 PM
Public Statistics
Date Registered ‎02-19-2018 11:15 PM
Last Visited ‎02-20-2018 11:32 PM
Total Messages Posted 2