Support Questions

Find answers, ask questions, and share your expertise

NiFi Scheduling - Event Driven vs Timer Driven Scheduling Strategy

avatar

I am newbie to NiFi, so excuse me for a very basic question!

My question is regarding how NiFi handles the processor executions. If I use Timer Driven scheduling strategy and keep run schedule and run duration as ZERO, how the behavior is different than Event Driven scheduler?

The definition of Timer Driven scheduling states that "The default value of 0 sec means that the Processor should run as often as possible as long as it has data to process."

Correct me if my understanding is wrong, the processor will keep on looking for events and will run as soon as there are some events available to be processed. Isn't it similar to Event Driven scheduler that it will keep on looking for events and will process them whenever they are available?

1 ACCEPTED SOLUTION

avatar
Master Mentor
@Rahul Soni

Event Driven Scheduling strategy is considered experimental. It was developed long ago as an alternative to Timer Driven scheduling strategy. With enhancements to the Timer Driven strategy since then, The event Driven strategy has pretty much been abandoned. It only still exists for now to avoid breaking any backwards capability for NiFi users.

Timer Driven strategy:

Processors have a configured schedule they run at. The 0 sec default means that the processor will attempt to run as often as possible. This works in conjunction with the concurrent task property. When the processor runs it use the concurrent task. The processor will be unable to execute again until that thread completes unless their are additional free threads available via additional configured concurrent tasks. Back when the event driven strategy was developed the timer driven strategy was not a s mature as it is now. With a run schedule of 0 secs and no work to do, the processor would end up consuming 100% of a cpu all the time constantly checking for work. The Timer Driven strategy has since had several improvements including yielding when there is no work to do to prevent excessive CPU usage.

Event Driven Strategy:

Rather then the processor deciding when to run, The NiFi controller would hand out threads based on work needing to be done. Essentially if an incoming connection to a processor using Event Driven strategy contained FlowFiles, the core would schedule that processor to execute. The concurrent tasks setting takes on a different meaning here. It is the ceiling more the maximum number of event driven threads that can be assigned concurrently to this processor. 0 would be very dangerous as it remove the ceiling al together. The strategy in the long run ended up being less performant then the Timer Driven strategy.

Thanks,

Matt

View solution in original post

2 REPLIES 2

avatar
Master Mentor
@Rahul Soni

Event Driven Scheduling strategy is considered experimental. It was developed long ago as an alternative to Timer Driven scheduling strategy. With enhancements to the Timer Driven strategy since then, The event Driven strategy has pretty much been abandoned. It only still exists for now to avoid breaking any backwards capability for NiFi users.

Timer Driven strategy:

Processors have a configured schedule they run at. The 0 sec default means that the processor will attempt to run as often as possible. This works in conjunction with the concurrent task property. When the processor runs it use the concurrent task. The processor will be unable to execute again until that thread completes unless their are additional free threads available via additional configured concurrent tasks. Back when the event driven strategy was developed the timer driven strategy was not a s mature as it is now. With a run schedule of 0 secs and no work to do, the processor would end up consuming 100% of a cpu all the time constantly checking for work. The Timer Driven strategy has since had several improvements including yielding when there is no work to do to prevent excessive CPU usage.

Event Driven Strategy:

Rather then the processor deciding when to run, The NiFi controller would hand out threads based on work needing to be done. Essentially if an incoming connection to a processor using Event Driven strategy contained FlowFiles, the core would schedule that processor to execute. The concurrent tasks setting takes on a different meaning here. It is the ceiling more the maximum number of event driven threads that can be assigned concurrently to this processor. 0 would be very dangerous as it remove the ceiling al together. The strategy in the long run ended up being less performant then the Timer Driven strategy.

Thanks,

Matt

avatar

Very well documented, Matt! Thanks 🙂