I'm part of a sysadmin team and we are providing Nifi enabled machines to many to our on-site development teams. We do not have control of the flows that will be developed in each one. However we would like to know if there is a flow consuming too much resources without interfering on the flow development process. Is there a way to monitor the machine to point to excessive consuming flows?
NiFi executes within a single JVM and therefore only one resource pool exists from which all dataflows get resources from. So from a standpoint of which thread from which dataflow is consuming the buck of resources, there is no endpoint to get those details. An updateAttribute processor running in dataflow A looks no different than an updateAttribute processor running in dataflow B from a thread standpoint.
That being said, there is some indirect information you can collect from the NiFi summary UI.
Processors report stats on Tasks/Time (5 minute stat) that can tell you if a given processor is consuming more CPU time than other components. You can also look at "Read /Write" stats that will tell you which processors are reading and writing the most bytes of data to the NiFi content repository disk.
You can get this summary information in a couple ways:
1. Create an external script that use the NiFi Rest-api to retrieve these details and then via your script parse the large json response for the specific stats you are looking for.
2. Create a new logger in the NiFi logback.xml and utlilize the NiFi ControllerStatusReportingTask to output these stats to your new logger. Then you can have a script parse these logs for the stats you care about.
Hope this helps you,