@hegdemahendra
All the dataflow(s) you construct in the NiFi UI and all the controller services reporting tasks, and templates you create all reside within your NiFi's heap memory. So the more you build the more heap that is being consumed there. You also end up with more components being scheduled. A scheduled processor will need a thread to check it's inbound connection or connect to an external service to check for new FlowFile or data to consume/execute upon. All these components then need to share the available configured threads from NiFi thread pool. A running processor is not idle. It still has a configured run schedule to which it uses to check for work (granted that with n. work to do those threads are extremely short lived). The default size of the Timer Driven Thread pool is only 10.
So as your flow gets larger, I'd recommend looking at Garbage Collection (GC) stats for your NiFi's JVM to see how often GC is happening and how long those GC events take (All GC is stop the world, so no processing happens during GC). I'd also examine your CPU load average and is it is low, increase the size of the Max Timer Driven Thread pool (Global menu --> controller settings --> general) in small increments to see what impact that has on yoru performance.
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt
