Support Questions
Find answers, ask questions, and share your expertise

How healthy is my Nifi installation? Need help with monitoring and understanding limits

Explorer

We've been using Nifi quite a lot recently to do a huge variety of tasks we never though possible without a dedicated software dev team - really fantastic product.

 

By now we have a lot of workflows going on, ranging from http API requests or simply doing small tasks related to data management.

 

I use Grafana to monitor my Nifi instance, but one thing that is bothering me is that I do not know how "healthy" my instance is and if or when I will start getting into trouble.

 

Here is what I currently monitor on my Nifi instance:

Tokolosk_0-1623239783421.png

(I hope this screenshot can be expanded)

 

What triggered this question is a new process that increased CPU usage from an average of 2% to 10%, and shortly after that Nifi rebooted automatically.

 

I'm going through the logs today to see what caused it - if I can find out why - but none of the graphs seemed to indicate an impending reboot. From observing the graphs, it seems everything is well managed: CPU load is small, Heap Memory gets cleaned up (most of the time hovering around 20-40%) and disk space is all good.

 

What should I be adding or looking out for to monitor Nifi better and have an understanding when one thing or another needs an upgrade? Or to know when I reached a limit to the number of processors or threads?

 

Tokolosk_1-1623240183732.png

Thank you!

 

Edit: just to add this is not my normal job expertise  - my background is quite different and I'm mostly learning this on the job.  So hopefully this is not too silly a question.

0 REPLIES 0