NIFi Error when creating a paramater: org.apache.nifi.web.util.LifecycleManagementException: Failed while waiting for components to transition to state of STOPPED

[#1]

I run into the below error when creating a paramater. This is NiFi version 1.11.4. Any thoughts?

Unable to complete parameter context update request: Failed to update Parameter Context due to org.apache.nifi.web.util.LifecycleManagementException: Failed while waiting for components to transition to state of STOPPED

[#2]

@SandeepG01 

What the ERROR is telling you is that some component the references the parameter context failed to transition to a stopped state.

When a parameter context is defined in a component (NiFi processor for example), and change to the value assigned to that parameter would require the restart of those components so that the new parameter value is read in by the component.

When a component like a NiFi processor is asked to stop, it first transitions into a state of "Stopping".  Once any active threads on that component complete the processor transitions to a "Stopped" state.  It as this point the component could be started again.

So there are two possible scenarios that can happen here Since NiFi will not interrupt an active thread:
1. A component had a ling running thread that the restart process timed out waiting for it to transition to stopped (thread to complete and exit).  It is quite possible that this component did finally finish its  thread execution post this timeout and another attempt to apply the parameter context change would not be successful.
2. A component has a hung thread.  Meaning the active thread on the component is making no progress because it is either waiting indefinitely on some response action or it being blocked by some other thread.  In this case it would never complete its execution and transition to the "Stopped"state.  First you need to identify the processor with the hung thread (Processors show a small number in t he upper right corner when a thread is active).  Then you can take a series of thread dumps (./nifi.sh dump <dump-file-01>) and compare those thread dumps to see if the you have a thread stuck in the same state in every dump (shows no thread progress).  The only way to kill a hung thread is via a NiFi restart.   NiFi processors offer a terminate option, but that does not kill the tread.  It simply disassociates that thread with the processor (think of putting in an isolation box). The flowfile that was being processed by that thread is rolled back on to the incoming connection.  Should that terminated thread ever return anything (lets say it was not hung, but just a very long running thread), it output is just sent to null since it was terminated.  But again that thread if truly hung will not go away until NiFi restart even if you selected terminate on the component.

You many also inspect the nifi-app.log for the ERROR exception to see if it is followed with a stack trace that may give more insight.

Hope this helps,

Matt

[#3]

I have a user getting this error as well, not finding much about it. These are brand new contexts, trying to create a first parameter. It gives me (administrator) no problems, but the user is apparently missing a policy, or something (bug?)

[#4]

@Seedy 

Sharing screenshots of the steps followed and observations seen would be helpful.

So you have created a Process Group (PG) and created a new Parameter Context which is then assigned to that PG.   Is this a nested PG?  The blank canvas you see when NiFi is first accessed is the root level PG.  Any PG you add to canvas is a nested/sub-PG to that root PG.   You can also have PGs nested even deeper.  As the user who is having issues, sharing screenshots may help in seeing what the issue may be.

Please include screenshots showing following while authenticated as user having issues:
1. Actions attempting to be performed
2. NiFi Parameter contexts UI (found under the NiFi global menu in upper right corner --> Parameter Contexts)

3. On the PG on which the parameter context has been assigned, show the configuration for that PG including the Controller Services tab.
Example:

4. Configuration of the assigned parameter context.  click through each existing parameter in the context, do any show "unauthorized referencing components"?

Thanks,
Matt

[#5]

I said in the question, there are no other parameters, until I created one, and there are no components referencing any parameters, because I just created the context. There aren't any other contexts, no components are using parameters, because it won't allow the uses to create them.

Since I saw a question with the same version and error code, was kind of hoping someone had a "Oh you forgot to << missing step >> when ..." type answer. 🙂

[#6]

The full info from the app log is:

2021-05-17 14:57:12,166 INFO org.apache.nifi.web.api.ParameterContextResource: Stopping 0 Processors in order to update Parameter Context
2021-05-17 14:57:12,173 ERROR org.apache.nifi.web.api.ParameterContextResource: Failed to update Parameter Context
org.apache.nifi.web.util.LifecycleManagementException: Failed while waiting for components to transition to state of STOPPED
        at org.apache.nifi.web.util.ClusterReplicationComponentLifecycle.scheduleComponents(ClusterReplicationComponentLifecycle.java:133)
        at org.apache.nifi.web.api.ParameterContextResource.stopProcessors(ParameterContextResource.java:960)
        at org.apache.nifi.web.api.ParameterContextResource.updateParameterContext(ParameterContextResource.java:852)
        at org.apache.nifi.web.api.ParameterContextResource.lambda$submitUpdateRequest$17(ParameterContextResource.java:804)
        at org.apache.nifi.web.api.concurrent.AsyncRequestManager$2.run(AsyncRequestManager.java:117)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

[#7]

@Seedy 

Sorry, I feel like some important details are missing here that would help in giving you the most thorough answer. 

I understand that your Admin user has no issues creating a parameter context and populating that parameter context with parameters; however, a non admin user is having issues.

1. What permissions have been granted to this user?
2. Where is the user going to create parameter context? By right clicking on Process Group (PG) --> configure --> Process Group Parameter Context --> Create new parameter context OR NiFi Global menu (upper right corner of UI) --> Parameter Contexts --> + (click plus icon to create new parameter context).

If you doing through configure on a PG, while adding new parameter context (before hitting apply), can you add a parameter first.  Does that Parameter stay without exception?   Does exception only occur when trying to add a parameter to a parameter context that has already been created?

There have been numerous issues addressed with parameter contexts between Apache NiFi 1.11.4 and 1.13.2.

It appears you may be hitting https://issues.apache.org/jira/browse/NIFI-7725
While I can generate same exception in 1.11.4 (just not sure if i followed you exact same path to do so), I do not get that same exception in 1.12.1.  

I recommend upgrading to the latest NiFi release.
Earlier is was asking about PG and sub-PG along with controller services assigned (Do not need any parameters referenced in these Controller services) because of another issue I had seen that affected being able to modify and add parameters to a parameter context for non admin users.

This issue was tracked under https://issues.apache.org/jira/browse/NIFI-8419
Above is fixed for the future 1.14 release.

So while upgrading may solve your specific issue here, you may hit this other issue, so I was trying to collect all the details to give you the best and most accurate help.

Thank you,

Matt

[#8]

Yes, that screenshot looks like what I get when logged in via a user account. After you add a parameter/value and hit apply, one green check then red X's the rest of the way down.
I had the AD folks create me a test user and have been trying all sorts of combinations of permissions, and they all have resulted in the same error.
Mostly just parameter context and controller related permissions, the user group has permissions to the PC in question, and I even gave the group permission to create new PC, but they still couldn't create a parameter.
We installed cfm 1.1.0.0 on one cluster, and 2.0.4.3 on another, both indicate NiFi 1.11.4, I will have to look into patch/upgrade paths, these are both new installs, different versions of cloudera.

[#9]

@Seedy 

Unfortunately, there is not CFM Private cloud release yet that includes NIFI-8419.
The latest CFM Private Cloud release is CFM 2.1.1 and includes everything in Apache NiFi 1.13.2 plus some additional bug fixes listed here:
https://docs.cloudera.com/cfm/2.1.1/release-notes/topics/cfm-nifi-patches.html

I strongly encourage you to upgrade to CFM 2.1.1 because of a number of bug fixes and some awesome improvements. You can file a support case asking for a hotfix that includes the bugs I mentioned, but i recommend asking for that hotfix post upgrade to CFM 2.1.1.  Product improvements and new features don't come in hotfixes.

Only way to get around this bug is making sure your restricted user has read and modify on the root process groups which is far from awesome.

Glad I could help identify the issue you have encountered.  Please take a moment to login and click accept on all answers you found assisted you with this query.

Matt

[#10]

Hmm, don't remember seeing the 'private cloud' nomenclature before.
our new version might work (Cloudera Enterprise 7.2.4)

Cloudera Flow Management (CFM) 2.0.4.3

1.11.4.2.0.4.3-1 built 01/06/2021 23:14:00 CST

Tagged nifi-1.11.4-RC1

But most of our stuff (Cloudera Enterprise 6.3.3) might be limited to the 1.1 branch.
Cloudera Flow Management (CFM) 1.1.0.0

1.11.4.1.1.0.0-119 built 04/24/2020 19:39:59 CDT

Tagged nifi-1.11.1-RC1