Frequent Node Disconnects and Flow Synchronization Issues in NiFi 1.25 with3 node cluster , node-0 gets issue all the time

Sanga — Tue, 04 Nov 2025 19:43:54 GMT

I have nifi cluster hosted on AKS , its 3 node with memory 20G each node. node has dedicated pvcs standard lrs.

We are seeing occasionally once in 1/2 weeks nifi node-0 only disconnects with below error.

We are running full load ingestions frequently. but there is no pattern . we run 3-4 processor groups have overlapping time (parallel). but not all the time we get error . most of the runs on most of the day work . sometime we get below mentioned error. what we observed . node-0 first gets disconnected later when it tried to reconnect org.apache.nifi.controller.serialization.FlowSynchronizationException.

Any help possible ?

Error dump :

2025-11-01 06:05:19,513 WARN [Notify Cluster of Node Status Change-2] o.a.n.c.p.i.StandardClusterCoordinationProtocolSender Failed to send Node Status Change message to nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443

org.apache.nifi.cluster.protocol.ProtocolException: Failed to create socket due to: java.net.UnknownHostException: nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local

at org.apache.nifi.cluster.protocol.impl.StandardClusterCoordinationProtocolSender.createSocket(StandardClusterCoordinationProtocolSender.java:216)

at org.apache.nifi.cluster.protocol.impl.StandardClusterCoordinationProtocolSender.createSocket(StandardClusterCoordinationProtocolSender.java:204)

at org.apache.nifi.cluster.protocol.impl.StandardClusterCoordinationProtocolSender.access$000(StandardClusterCoordinationProtocolSender.java:67)

at org.apache.nifi.cluster.protocol.impl.StandardClusterCoordinationProtocolSender$2.run(StandardClusterCoordinationProtocolSender.java:296)

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.base/java.util.concurrent.FutureTask.run(Unknown Source)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.base/java.lang.Thread.run(Unknown Source)

Caused by: java.net.UnknownHostException: nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local

at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source)

at java.base/java.net.SocksSocketImpl.connect(Unknown Source)

at java.base/java.net.Socket.connect(Unknown Source)

at java.base/sun.security.ssl.SSLSocketImpl.connect(Unknown Source)

at java.base/sun.security.ssl.SSLSocketImpl.<init>(Unknown Source)

at java.base/sun.security.ssl.SSLSocketFactoryImpl.createSocket(Unknown Source)

at org.apache.nifi.io.socket.SocketUtils.createSocket(SocketUtils.java:68)

at org.apache.nifi.cluster.protocol.impl.StandardClusterCoordinationProtocolSender.createSocket(StandardClusterCoordinationProtocolSender.java:210)

... 8 common frames omitted

2025-11-01 06:05:18,438 INFO [Heartbeat Monitor Thread-1] o.a.n.c.c.h.AbstractHeartbeatMonitor Finished processing 2 heartbeats in 20659 nanos

2025-11-01 06:05:18,463 INFO [NiFi Web Server-20322] o.a.n.c.c.node.NodeClusterCoordinator Event Reported for nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443 -- Requesting that node connect to cluster on behalf of username

2025-11-01 06:05:18,464 INFO [NiFi Web Server-20322] o.a.n.c.c.node.NodeClusterCoordinator Status of nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443 changed from NodeConnectionStatus[nodeId=nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443, state=DISCONNECTED, Disconnect Code=Node's Flow did not Match Cluster Flow, Disconnect Reason=org.apache.nifi.controller.serialization.FlowSynchronizationException: Failed to connect node to cluster because local flow controller partially updated. Administrator should disconnect node and review flow for corruption., updateId=44] to NodeConnectionStatus[nodeId=nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443, state=CONNECTING, updateId=5]

2025-11-01 06:05:18,492 WARN [Notify Cluster of Node Status Change-2] o.a.n.c.p.i.StandardClusterCoordinationProtocolSender Failed to send Node Status Change message to nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443

org.apache.nifi.cluster.protocol.ProtocolException: Failed to create socket due to: java.net.UnknownHostException: nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local

Re: Frequent Node Disconnects and Flow Synchronization Issues in NiFi 1.25 with3 node cluster , node-0 gets issue all the time

MattWho — Thu, 04 Dec 2025 13:55:38 GMT

@Sanga

What reason is being logged by the elected cluster coordinator node when node-0 gets initially disconnected? Is it a lack of heartbeat?

I understand that onc it gets initially disconnected, NiFi cluster coordinator requests that it reconnect on next heartbeat and the triggered flow synchronization result in a exception blocking the node from rejoining the cluster.

My initial thought is that you may be hitting NIFI-12969 or NIFI-12232 which was addressed in Apache NiFi 1.26. An upgrade may help with your issue.

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

question Re: Frequent Node Disconnects and Flow Synchronization Issues in NiFi 1.25 with3 node cluster , node-0 gets issue all the time in Support Questions

Frequent Node Disconnects and Flow Synchronization Issues in NiFi 1.25 with3 node cluster , node-0 gets issue all the time

Re: Frequent Node Disconnects and Flow Synchronization Issues in NiFi 1.25 with3 node cluster , node-0 gets issue all the time