Nifi admin rest api becomes slow day by day if nifi service not restarted

[#1]

Hello Experts,

We are using Nifi for continuous data ingestion over http port (as rest api) and also we use nifi admin rest api to dynamically start/stop procesors or update variable registry or create process groups etc.

We are facing a challenge with these admin rest api's (/nifi-api), i.e   when nifi restarted all apis responds very quickly and works fine and when days go on (if no nifi restart in btwn) they nifi rest api's becomes very slow and at some point they stop responding or timing out.

And this slowness/not responding issue will resolve only after nifi restart.

Since we have real time data ingestion via http port , we cant really restart frequently.

Nifi version : 1.16.3

Is there any solution for this problem ? or any suggestion to tackle this would be much appreciated

Thanks,

Mahendra

[#2]

@hegdemahendra   This instability is usually a result of the cluster's tuning and/or workload greater than cluster tunning and spec can sustain for long period of time.   When you restart the cluster, you are effectively resetting the memory allocation.   If you are restarting the entire nifi server(s), then you are giving a fresh operating system just like rebooting windows if its running slow.  Things will work fine again until cross the fine line between working and not working.

Before restart, and making tuning steps, you should document the state of the system when it is not responding as expected.  What is the load on each node?  What is the used/vs available memory? etc.   Next you should investigate Garbage Collection.

[#3]

@hegdemahendra 

You also need to be careful with some rest-api request as they may generate reports that are held in NiFi heap until a rest-api call is made to remove them.  Doing similar actions directly from the UI handles the multiple calls needed automatically.  So calls like these can slowly eat away at the NiFi heap impacting performance until a restart.  Provenance queries would be an example of this.

Unrelated note:  NiFi Variable registry has been deprecated in favor of NiFi Parameters now.  The Variable registry functionality has been officially removed starting with the latest NiFi 2.0.0-M1 release.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt