With a large number of processors and connections, NiFi boots up very slowly.

Soli — Mon, 17 Nov 2025 02:03:41 GMT

Current Issue:
Recently, we have encountered a significant slowdown during NiFi cluster restarts. The startup process takes approximately 10 to 15 minutes, and the `nifi-app.log` indicates that NiFi is spending most of this time restoring components. For example, we see many log entries like:

2025-11-15 15:13:51,911 INFO [main] o.a.n.f.s.StandardVersionedComponentSynchronizer Added Connection[ID=70a5878d-0199-1000-ffff-ffff9a65a5f2, Source ID=7097063b-0199-1000-0000-0000565ddd06, Dest ID=7096bae9-0199-1000-ffff-ffffbae5c592] to StandardProcessGroup[identifier=70967de1-0199-1000-0000-00001dde7b99,name=Filter empty files]
2025-11-15 15:13:51,913 INFO [main] o.a.n.f.s.StandardVersionedComponentSynchronizer Added Connection[ID=7097150c-0199-1000-ffff-ffffd3ea7bb1, Source ID=7096aedf-0199-1000-0000-000061416339, Dest ID=7097063b-0199-1000-0000-0000565ddd06] to StandardProcessGroup[identifier=70967de1-0199-1000-0000-00001dde7b99,name=Filter empty files]
2025-11-15 15:13:51,914 INFO [main] o.a.n.f.s.StandardVersionedComponentSynchronizer Added Connection[ID=70a4170c-0199-1000-0000-00000c861b3a, Source ID=70a1808b-0199-1000-0000-000010d88e3d, Dest ID=92923d85-ec54-12f9-97aa-43ac5442d61c] to StandardProcessGroup[identifier=70967de1-0199-1000-0000-00001dde7b99,name=Filter empty files]
2025-11-15 15:13:51,915 INFO [main] o.a.n.f.s.StandardVersionedComponentSynchronizer Added Connection[ID=70a16555-0199-1000-0000-00002007ddf8, Source ID=7097063b-0199-1000-0000-0000565ddd06, Dest ID=70a1808b-0199-1000-0000-000010d88e3d] to StandardProcessGroup[identifier=70967de1-0199-1000-0000-00001dde7b99,name=Filter empty files]
2025-11-15 15:13:51,915 INFO [main] o.a.n.f.s.StandardVersionedComponentSynchronizer Added Connection[ID=70a3f911-0199-1000-0000-0000498a8368, Source ID=70a1808b-0199-1000-0000-000010d88e3d, Dest ID=7096bae9-0199-1000-ffff-ffffbae5c592] to StandardProcessGroup[identifier=70967de1-0199-1000-0000-00001dde7b99,name=Filter empty files]
2025-11-15 15:13:51,916 INFO [main] o.a.n.f.s.StandardVersionedComponentSynchronizer Added Connection[ID=70a44583-0199-1000-ffff-ffffc73bccdb, Source ID=b4f138b4-271e-16af-b734-a2005561c5b3, Dest ID=b4f138b4-271e-16af-b734-a2005561c5b3] to StandardProcessGroup[identifier=70967de1-0199-1000-0000-00001dde7b99,name=Filter empty files]

Our Environment:
- NiFi Version: 1.22.0 (3-node cluster mode)
- Java Version: jdk-11.0.16.1
- Operating System: Red Hat 8.10
- CPU: 32 cores
- Memory: 48GB

Our Flow Summary:
- Processors: 20,725
- Connections: 34,000
- Process Groups: 217
- Output Ports: 215

Re: With a large number of processors and connections, NiFi boots up very slowly.

MattWho — Mon, 17 Nov 2025 14:48:57 GMT

@Soli

Unfortunately, there is not enough shared to say what is taking so long during your startup. Above log snippet shows ~.001 millisecond between each log line. At 34,000 connections, that would be only 34 seconds. Assuming About the same for other components, I don't think the synchronizing of components is what is taking the bulk of the 10-15 minutes of startup time.

In the nifi-app.log, what time is reported application start:

INFO [main] org.apache.nifi.runtime.Application Started Application in <xxxxx> seconds

What does each node report here? Are all nodes taking roughly same time to start or is there any specific node taking longer then the other two?

Is performance good once NiFi is up and running?
Are all ~21,000 components in the running state?

Any WARN or ERROR logging during startup?

How many FlowFiles are queued per node?

After NiFi is started, maybe collect a verbose NiFi diagnostics output for review.

./bin/nifi.sh diagnostics --verbose diag.txt

You may also collect thread dumps every minute during startup to see what thread during is taking so long.

Hopefully some of this will help you see what is slowing your startup.

There have been some improvements to startup time in Apache NiFi 2.x release.
There were a few improvements to startup in Nifi 1.x releases, but those were made prior to 1.22.

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

question Re: With a large number of processors and connections, NiFi boots up very slowly. in Support Questions

With a large number of processors and connections, NiFi boots up very slowly.

Re: With a large number of processors and connections, NiFi boots up very slowly.