Created 12-22-2017 04:48 AM
Hi,
I am configured HDF 3.0.1 with 3 nodes : HDF1 (Primary), HDF2 and HDF3, There was Out of Disk space issues on HDF1 and HDF3, after recovery HDF1 is not starting up with below error.
nifi-app.log
2017-12-21 16:37:47,462 ERROR [main] o.a.nifi.controller.StandardFlowService Failed to load flow from cluster due to: org.apache.nifi.controller.UninheritableFlowException: Failed to connect node to cluster because local flow is different than cluster flow. org.apache.nifi.controller.UninheritableFlowException: Failed to connect node to cluster because local flow is different than cluster flow. at org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:936) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:515) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:800) at org.apache.nifi.NiFi.<init>(NiFi.java:160) at org.apache.nifi.NiFi.main(NiFi.java:267) Caused by: org.apache.nifi.controller.UninheritableFlowException: Proposed configuration is not inheritable by the flow controller because of flow differences: Found difference in Flows: Local Fingerprint: 7c84501d-d10c-407c-b9f3-1d80e38fe36a3b80ba0f-a6c0-48db-b721-4dbc04cef28eorg.apache.nifi.reporting.ambari.AmbariReportingTaskAmbariReport Cluster Fingerprint: 7c84501d-d10c-407c-b9f3-1d80e38fe36a826ffb50-015e-1000-ffff-ffffd8f61569org.apache.nifi.processors.standard.ExtractTextNO_VALUEorg.apach at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:259) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1576) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:84) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:722) at org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:911) ... 4 common frames omitted 2017-12-21 16:37:47,463 INFO [main] o.a.n.c.c.node.NodeClusterCoordinator HDF1:9090 requested disconnection from cluster due to org.apache.nifi.controller.UninheritableFlowException: Failed to connect node to cluster because local flow is different than cluster flow. 2017-12-21 16:37:47,463 INFO [main] o.a.n.c.c.node.NodeClusterCoordinator Status of HDF1:9090 changed from NodeConnectionStatus[nodeId=HDF1:9090, state=CONNECTING, updateId=7] to NodeConnectionStatus[nodeId=HDF1:9090, state=DISCONNECTED, Disconnect Code=Node's Flow did not Match Cluster Flow, Disconnect Reason=org.apache.nifi.controller.UninheritableFlowException: Failed to connect node to cluster because local flow is different than cluster flow., updateId=7] 2017-12-21 16:37:47,482 ERROR [main]
o.a.n.c.c.node.NodeClusterCoordinator Event Reported for HDF1:9090 -- Node disconnected from cluster due to org.apache.nifi.controller.UninheritableFlowException:Failed to connect node to cluster because local flow is different than cluster flow. 2017-12-21 16:37:47,482 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager Cannot unregister Leader Election Role 'Primary Node' becuase that role is not registered 2017-12-21 16:37:47,483 WARN [main]
org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. java.lang.IllegalStateException: Already closed or has not been started at com.google.common.base.Preconditions.checkState(Preconditions.java:173) at org.apache.curator.framework.recipes.leader.LeaderSelector.close(LeaderSelector.java:270) at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.unregister(CuratorLeaderElectionManager.java:151) at org.apache.nifi.controller.FlowController.setClustered(FlowController.java:3744) at org.apache.nifi.controller.StandardFlowService.handleConnectionFailure(StandardFlowService.java:554) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:518) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:800) at org.apache.nifi.NiFi.<init>(NiFi.java:160) at org.apache.nifi.NiFi.main(NiFi.java:267) 2017-12-21 16:37:47,484 INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server... 2017-12-21 16:37:47,496 INFO [Thread-1] o.eclipse.jetty.server.AbstractConnector Stopped ServerConnector@672c6b93{HTTP/1.1,[http/1.1]}{HDF1:9090} 2017-12-21 16:37:47,497 INFO [Thread-1] org.eclipse.jetty.server.session Stopped scavenging
Created 12-22-2017 03:30 PM
Shut everything down and copy flow.xml.gz from one of the other nodes to HDF1 which will get HDF1 back in sync.
If you are running securely and using NiFI's file-based authorizer, then you will also want to copy users.xml and authorizations.xml from one of the other nodes to HDF1, because those are also factored into joining the cluster.