@AndreyN
Each processor by default uses Timer Driven Scheduling strategy and a run Schedule of 0 secs. This means that each processor is constantly requesting threads from the Max Timer Driven Thread pool and checking for work (work being any FlowFiles on inbound connections or in case of an ingest processor, connecting to that ingest point whether local dir or remote service to check for data). While generally these checks for work take micro seconds or longer depending on processor, NiFi does have a global setting for yielding the processors when the previous run resulted in no FlowFiles processed/produced.
To prevent excessive latency this back duration by default is very short (10 ms).
To adjust this setting, you can change the following property in the nifi.properties file:
nifi.bored.yield.duration found in Core Properties section of admin guide.
Keep in mind that this setting impacts all processors. So higher you set the more latency that could exist before new work is found after a run that resulted in no work.
You can also selectively adjust the run schedule on select processors. 0 sec run schedule means to run as fast as possible. So as soon as one thread completes, request another (unless thread results in no work wait bored duration before requesting next thread). So if you have flows that are always very light and latency is not a concern, you could set those processors to only get scheduled to execute every 1 sec or longer.
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt
@AndreyN
Each processor by default uses Timer Driven Scheduling strategy and a run Schedule of 0 secs. This means that each processor is constantly requesting threads from the Max Timer Driven Thread pool and checking for work (work being any FlowFiles on inbound connections or in case of an ingest processor, connecting to that ingest point whether local dir or remote service to check for data). While generally these checks for work take micro seconds or longer depending on processor, NiFi does have a global setting for yielding the processors when the previous run resulted in no FlowFiles processed/produced.
To prevent excessive latency this back duration by default is very short (10 ms).
To adjust this setting, you can change the following property in the nifi.properties file:
nifi.bored.yield.duration found in Core Properties section of admin guide.
Keep in mind that this setting impacts all processors. So higher you set the more latency that could exist before new work is found after a run that resulted in no work.
You can also selectively adjust the run schedule on select processors. 0 sec run schedule means to run as fast as possible. So as soon as one thread completes, request another (unless thread results in no work wait bored duration before requesting next thread). So if you have flows that are always very light and latency is not a concern, you could set those processors to only get scheduled to execute every 1 sec or longer.
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt
