Member since
11-18-2021
2
Posts
0
Kudos Received
0
Solutions
11-22-2021
09:26 AM
@ashinde, thank you very much for the response. Makes sense why that setting is not working for us on 1.11.4. I will have to check when we can be on 1.12 version to use some of the additional features that'd help us with the troubleshooting. Point taken about finding the actual RC for the heartbeat delay. CPU, and Memory (GC, heap usage) have not been an issue so far during our investigations. We are not really running any high volume processing here, and we are on a cluster with 3 nodes each with 24 cores, and 22 GB heap allocation. Network is one thing we need to check further, but so far basic analysis does not show any delay b/w the nodes or any packet loss. For now we have stopped the frequent node disconnections due to missing heartbeats by increasing nifi.cluster.protocol.heartbeat.interval=30 sec, while we investigate the issue further. Best, MT13
... View more
11-18-2021
09:14 AM
Hi, We are on nifi-1.11.4, on a 3 node cluster, and we are observing frequent node disconnections. ..Disconnect Code=Lack of Heartbeat , Disconnect Reason=Have not received a heartbeat from node in 42 seconds , updateId=62] While we investigate the issue for lack of heartbeat, we were trying to increase the disconnection threshold from the default 40 seconds [nifi.cluster.protocol.heartbeat.missable.max=8 X nifi.cluster.protocol.heartbeat.interval=5 sec] to 300 seconds. However, even when we applied the nifi.cluster.protocol.heartbeat.missable.max=60 setting change on all nodes, we still observe the node disconnections occurring after 40 seconds. Any thoughts on why this setting is not getting picked up? nifi.cluster.protocol.heartbeat.interval=5 sec nifi.cluster.protocol.heartbeat.missable.max=60 Thank you -MT13
... View more
Labels:
- Labels:
-
Apache NiFi