GenerateFlowFile "Failed to properly initialize Processor. If still scheduled to run"

Bern — Tue, 07 Jan 2025 20:29:05 GMT

Dear Sirs, I have the following problem with my nifi cluster, it takes a long time to run a flow and when it does it presents the following error:

Failed to properly initialize Processor. If still scheduled to run,
NiFi will attempt to initialize and run the Processor again after the 'Administrative Yield Duration' has elapsed.
Failure is due to org.apache.nifi.processor.exception.TerminatedTaskException: org.apache.nifi.processor.exception.TerminatedTaskException

Nifi version: 1.11.4
Zookeeper: external, apache-zookeeper-3.6.3

I performed the following actions seen in this previous post:
https://community.cloudera.com/t5/Support-Questions/Apache-Nifi-ListSFTP-Processor-Failed-to-properly-initialize/m-p/366854

My state-managemnet.
<cluster-provider>
<id>zk-provider</id>
<class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
<property name="Connect String">node-1:2181,node-2:2181,node-3:2181</property>
<property name="Root Node">/nifi</property>
<property name="Session Timeout">10 seconds</property>
<property name="Access Control">Open</property>
</cluster-provider>

and the zookeeper properties with the following:
server.1=node-1:2888:3888;2181
server.2=node-2:2888:3888 ;2181
server.3=node-3:2888:3888;2181

Please I need support, I waste a lot of time testing and implementing flows because of this problem

Re: GenerateFlowFile "Failed to properly initialize Processor. If still scheduled to run"

MattWho — Mon, 03 Mar 2025 14:04:01 GMT

@Bern

Unfortunately there is not enough information here to understand exactly what is going on.

The only exception shared was related to an attempt to terminate a thread on some processor. As far as why you see this, there is not enough information to say. It could be a bug in an older version, could be load issue, could be thread pool exhaustion, etc.

Observations and questions:

You are running with a very old version of Apache NiFi release 6+ years ago and one of the first releases to offer the Load-Balanced connections feature which was very buggy when it first was introduced. You would greatly benefit from upgrading for security fix and bug fixes reason.
You see to be using the load-balanced connections excessively. It makes sense to redistribute NiFi FlowFiles in connections after your executeSQL processors, but i see no value in redistributing after RouteOnAttribute or on the failure connections. This just adds excessive and unnecessary network traffic load.
I see you have ~1400 running components and a queue of ~265,000 FlowFiles. What is the CPU load average on each of yoru nodes and how many nodes do you have in your NiFi cluster?
What java version is being used? Are Garbage Collection (GC) stats healthy. How often is GC (partial and full) running? How long is spent on GC?
Any other ERROR in your nifi-app.log?
Have you taken any thread dumps when you are having issues with processor component threads? What did you observe there?

Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

question Re: GenerateFlowFile "Failed to properly initialize Processor. If still scheduled to run" in Support Questions

GenerateFlowFile "Failed to properly initialize Processor. If still scheduled to run"

Re: GenerateFlowFile "Failed to properly initialize Processor. If still scheduled to run"