Member since
07-15-2025
2
Posts
0
Kudos Received
0
Solutions
07-16-2025
03:05 AM
nifi.properties: nifi.cluster.protocol.heartbeat.interval=5 sec nifi.cluster.node.protocol.threads=10 nifi.cluster.node.protocol.max.threads=50 nifi.cluster.node.event.history.size=25 nifi.cluster.node.connection.timeout=2 mins nifi.cluster.node.read.timeout=2 mins nifi.cluster.node.max.concurrent.requests=150 nifi.cluster.firewall.file= nifi.cluster.flow.election.max.wait.time=1 mins nifi.cluster.flow.election.max.candidates= nifi.cluster.load.balance.connections.per.node=4 nifi.cluster.load.balance.max.thread.count=12 nifi.zookeeper.connect.timeout=30 secs nifi.zookeeper.session.timeout=30 secs zookeeper.properties: initLimit=10 autopurge.purgeInterval=24 syncLimit=5 tickTime=2000 dataDir=./state/zookeeper autopurge.snapRetainCount=30 these are the properties related to nifi and zookeeper We are seeing below errors in logs all the time Node disconnected due to Proposed Authorizer is not inheritable by the Flow Controller because NiFi has already started the dataflow and Authorizer has differences: Proposed Authorizations do not match current Authorizations: Proposed fingerprint is not inheritable because the current access policies is not empty. Failed to connect node to cluster because local flow controller partially updated. Administrator should disconnect node and review flow for corruption
... View more
07-15-2025
12:05 PM
Hi Cloudera Community, We are running Apache NiFi version 1.28.1 in a clustered setup with the following specifications: Cluster Size: 6 nodes Each Node: 32 vCPUs, 256 GB RAM JVM Heap Memory: 192 GB (configured per node) Max Timer Driven Thread Count: 192 Processor Count: Over 10,000 processors across the flows Java version 11 We are experiencing the following issues: Frequent node disconnections Flow synchronization failures during node reconnects Occasionally, policies appear empty when nodes rejoin We have ensured the flow.xml.gz, authorizations.xml, and users.xml files are consistent across all nodes. However, the issues still persist. Could you please advise: What could be causing these frequent node disconnects and flow sync failures? Is there an upper limit on the number of processors or thread count that could lead to instability? Are there recommended JVM GC or NiFi tuning settings for high-core, high-memory environments? Any insights or tuning recommendations would be greatly appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi