Support Questions
Find answers, ask questions, and share your expertise

How healthy is my Nifi installation? Need help with monitoring and understanding limits


We've been using Nifi quite a lot recently to do a huge variety of tasks we never though possible without a dedicated software dev team - really fantastic product.


By now we have a lot of workflows going on, ranging from http API requests or simply doing small tasks related to data management.


I use Grafana to monitor my Nifi instance, but one thing that is bothering me is that I do not know how "healthy" my instance is and if or when I will start getting into trouble.


Here is what I currently monitor on my Nifi instance:


(I hope this screenshot can be expanded)


What triggered this question is a new process that increased CPU usage from an average of 2% to 10%, and shortly after that Nifi rebooted automatically.


I'm going through the logs today to see what caused it - if I can find out why - but none of the graphs seemed to indicate an impending reboot. From observing the graphs, it seems everything is well managed: CPU load is small, Heap Memory gets cleaned up (most of the time hovering around 20-40%) and disk space is all good.


What should I be adding or looking out for to monitor Nifi better and have an understanding when one thing or another needs an upgrade? Or to know when I reached a limit to the number of processors or threads?



Thank you!


Edit: just to add this is not my normal job expertise  - my background is quite different and I'm mostly learning this on the job.  So hopefully this is not too silly a question.