I have 5 nodes of NiFi cluster. using 3000 processors. which contains 150+ pipelines(which have some simple use cases like reading the date from RDBMS or from file directory, applying some transformation, and writing it to Kudu table or HDFS. and in some use cases we are making http calls by using InvokeHTTP and JOLTTransformation processor). we are using PutKudu processor in many pipelines. some of the custom processors also we used(like ExecuteGroovyScript, GetSMPP). PFB the cluster details : Each node has 24 GB ram. 2 nodes have 16 core, another 2 nodes have 32 core and the remaining node have 48 cores. Almost all the times, I am not seeing much files in the queue and active threads are also like less than 50 in each node. but still we find the NiFi UI slow. and each node consuming more than 80% of jvm heap space regularly. CPU utilization are also going beyond 80% Can someone review my configuration or guide me here to resolve this issue.
... View more
