Member since
11-04-2025
2
Posts
0
Kudos Received
0
Solutions
11-05-2025
10:41 AM
@Sanga What is the specific Apache NiFi version being used? Any disk space issues on Node-0? Any other exceptions in the NiFi logs? Logs shared cover the reconnection attempt. What was the reason given for the node disconnection? Thank you, Matt
... View more
11-04-2025
11:43 AM
I have nifi cluster hosted on AKS , its 3 node with memory 20G each node. node has dedicated pvcs standard lrs. We are seeing occasionally once in 1/2 weeks nifi node-0 only disconnects with below error. We are running full load ingestions frequently. but there is no pattern . we run 3-4 processor groups have overlapping time (parallel). but not all the time we get error . most of the runs on most of the day work . sometime we get below mentioned error. what we observed . node-0 first gets disconnected later when it tried to reconnect org.apache.nifi.controller.serialization.FlowSynchronizationException. Any help possible ? Error dump : 2025-11-01 06:05:19,513 WARN [Notify Cluster of Node Status Change-2] o.a.n.c.p.i.StandardClusterCoordinationProtocolSender Failed to send Node Status Change message to nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443 org.apache.nifi.cluster.protocol.ProtocolException: Failed to create socket due to: java.net.UnknownHostException: nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local at org.apache.nifi.cluster.protocol.impl.StandardClusterCoordinationProtocolSender.createSocket(StandardClusterCoordinationProtocolSender.java:216) at org.apache.nifi.cluster.protocol.impl.StandardClusterCoordinationProtocolSender.createSocket(StandardClusterCoordinationProtocolSender.java:204) at org.apache.nifi.cluster.protocol.impl.StandardClusterCoordinationProtocolSender.access$000(StandardClusterCoordinationProtocolSender.java:67) at org.apache.nifi.cluster.protocol.impl.StandardClusterCoordinationProtocolSender$2.run(StandardClusterCoordinationProtocolSender.java:296) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.UnknownHostException: nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.connect(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.<init>(Unknown Source) at java.base/sun.security.ssl.SSLSocketFactoryImpl.createSocket(Unknown Source) at org.apache.nifi.io.socket.SocketUtils.createSocket(SocketUtils.java:68) at org.apache.nifi.cluster.protocol.impl.StandardClusterCoordinationProtocolSender.createSocket(StandardClusterCoordinationProtocolSender.java:210) ... 8 common frames omitted 2025-11-01 06:05:18,438 INFO [Heartbeat Monitor Thread-1] o.a.n.c.c.h.AbstractHeartbeatMonitor Finished processing 2 heartbeats in 20659 nanos 2025-11-01 06:05:18,463 INFO [NiFi Web Server-20322] o.a.n.c.c.node.NodeClusterCoordinator Event Reported for nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443 -- Requesting that node connect to cluster on behalf of username 2025-11-01 06:05:18,464 INFO [NiFi Web Server-20322] o.a.n.c.c.node.NodeClusterCoordinator Status of nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443 changed from NodeConnectionStatus[nodeId=nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443, state=DISCONNECTED, Disconnect Code=Node's Flow did not Match Cluster Flow, Disconnect Reason=org.apache.nifi.controller.serialization.FlowSynchronizationException: Failed to connect node to cluster because local flow controller partially updated. Administrator should disconnect node and review flow for corruption., updateId=44] to NodeConnectionStatus[nodeId=nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443, state=CONNECTING, updateId=5] 2025-11-01 06:05:18,492 WARN [Notify Cluster of Node Status Change-2] o.a.n.c.p.i.StandardClusterCoordinationProtocolSender Failed to send Node Status Change message to nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443 org.apache.nifi.cluster.protocol.ProtocolException: Failed to create socket due to: java.net.UnknownHostException: nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local
... View more
Labels:
- Labels:
-
Apache NiFi