Apache nifi-api is taking too long time to update variable registry

hegdemahendra — Wed, 01 Feb 2023 06:38:54 GMT

Hello Experts,

I am using nifi-api to update the variable registry of specific processor groups.

It was working as expected, but in recent time its taking too long (some time infinite) time to complete the variable registry update.
However if I restart nifi and try again this var registry update api it works quickly as expected.

Is it because of increased number/size of variable registry or any resource crunch ?
Could you please suggest a solution on this ?

Any suggestion is much appreciated in advance.

Thanks

Mahendra

Re: Apache nifi-api is taking too long time to update variable registry

MattWho — Thu, 02 Feb 2023 21:04:36 GMT

@hegdemahendra
When updating a NiFi variable a serious of steps needs to occur.

Steps To Update Variables

Identifying components affected
Stopping affected Processors
Disabling affected Controller Services
Applying Updates
Re-Enabling affected Controller Services
Restarting affected Processors

So in this process what can lead to this taking a long time to complete is the Stopping of processors and disabling of controller services using that updated variable. When NiFi requests a component to stop it transitions to a "stopping" or "disabling" stage. During this phase the component will not linger be scheduled and the process waits for any existing threads being executed by those components to complete. Those threads do not get interrupted.

So when this take aa long time or "infinite" amount of time, troubleshooting this would require getting a series of thread dumps to see which threads are long running or perhaps hung preventing the impacted components from competing the thread execution that blocks the component from transition to a fully stopped or disabled state. Understand that nothing in a thread dump is going to directly point back to a very specific processor. So it is important in your troubleshooting that you know what processors use the variable you are updating and look for threads that appear in the complete series of multiple thread dumps that relate back to those component classes.

Also in cases were you see these long running calls, are they for a variables used consistently by the same set of components to help narrow your analysis.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

question Re: Apache nifi-api is taking too long time to update variable registry in Support Questions

Apache nifi-api is taking too long time to update variable registry

Re: Apache nifi-api is taking too long time to update variable registry