I am using nifi in production environment and I need to run it in a 3-node cluster on docker (3 separate VMs).
Have used the following environment variables for Nifi:
- NIFI_ELECTION_MAX_WAIT=1 min
Through this method, I am able to scale up nifi on a single VM, but need to run multiple instances of this on other nodes and run the same in the same cluster.
When I use the property:
it renders the following stack:
2021-06-17 07:57:02,424 WARN [main] o.a.nifi.controller.StandardFlowService There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Cluster Coordinator and will attempt to connect to cluster again
2021-06-17 07:57:02,424 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Attempted to register Leader Election for role 'Cluster Coordinator' but this role is already registered
Any way to run nifi, zookeeper in 3 separate VMs in a cluster?
Thanks in advance!
Is it possible your nifi is configured to run the embedded zookeeper despite your trying to connect to your own zookeepers? Check if the nifi.state.management.embedded.zookeeper.start property in your nifi.properties file is set to true.
Check if you are able to communicate with all your zookeepers from each of your nodes. This definitely seems like an issue communicating with your ZKs.