Migrating HDF 2.0 node from standalone to cluster mode gives error.

[#1]

I have a HDF 2.0 node that was running in standalone mode. When I convert it to cluster mode, by changing nifi.properties file and restarting HDF 2.0, I am getting following error message.

Cluster is still in the process of voting on the appropriate Data Flow.

I have removed existing flow.xml.gz from conf directory and working with an empty flow file.

[#2]

is zookeeper running?

[#3]

Check in your nifi.properties, there are two properties:

nifi.cluster.flow.election.max.wait.time=5 mins

nifi.cluster.flow.election.max.candidates=

It is probably waiting 5 mins based on the first property. Usually you want to set the second property to a number <= to the nodes in your cluster, and then they will vote on the flow which will likely happen way faster than the 5 mins.

[#4]

My HDF just gave up suddenly. After restarting NiFi and later the zookeeper node, it gives the error "Invalid State. Cluster is still in the process of voting on the appropriate Data Flow." I have checked the properties suggested by @Bryan Bende, they are set as nifi.cluster.flow.election.max.candidates=1, nifi.cluster.flow.election.max.wait.time=5 mins. @Shishir Saxena I have also checked my /etc/hosts file and it's properly setup. Is there anything i'm not getting right?

[#5]

How many NiFi nodes are there? Can you see what is in /var/log/nifi/nifi-app.log for each of the NiFi nodes when this is happening?

[#6]

There's only one NiFi node @Bryan Bende. The nifi-app log does not suggest any error. I also notice that my flowfile-repo is filling up so fast.

2016-12-22 08:22:20,574 INFO [Timer-Driven Process Thread-13] o.a.n.controller.FileSystemSwapManager Successfully swapped out 10000 FlowFiles from FlowFileQueue[id=c5633890-4560-19cc-986a-56b900117c28] to Swap File /apps1/flowfile-repo/swap/1482387740171-c5633890-4560-19cc-986a-56b900117c28-4c2f21a9-23c1-494a-b92f-89b9a85c4f32.swap
2016-12-22 08:22:20,658 INFO [Timer-Driven Process Thread-13] o.a.n.c.r.WriteAheadFlowFileRepository Successfully swapped out 10000 FlowFiles from FlowFileQueue[id=c5633890-4560-19cc-986a-56b900117c28] to Swap File /apps1/flowfile-repo/swap/1482387740171-c5633890-4560-19cc-986a-56b900117c28-4c2f21a9-23c1-494a-b92f-89b9a85c4f32.swap
2016-12-22 08:22:24,803 INFO [Timer-Driven Process Thread-10] o.a.n.c.r.WriteAheadFlowFileRepository Repository updated to reflect that 10000 FlowFiles were swapped in to FlowFileQueue[id=c5633890-4560-19cc-986a-56b900117c28]
2016-12-22 08:22:25,647 INFO [Timer-Driven Process Thread-3] o.a.n.controller.FileSystemSwapManager Successfully swapped out 10000 FlowFiles from FlowFileQueue[id=c5633890-4560-19cc-986a-56b900117c28] to Swap File /apps1/flowfile-repo/swap/1482387745230-c5633890-4560-19cc-986a-56b900117c28-ea0d7a57-d319-4827-833e-ce94bfde1041.swap
2016-12-22 08:22:25,685 INFO [Timer-Driven Process Thread-3] o.a.n.c.r.WriteAheadFlowFileRepository Successfully swapped out 10000 FlowFiles from FlowFileQueue[id=c5633890-4560-19cc-986a-56b900117c28] to Swap File /apps1/flowfile-repo/swap/1482387745230-c5633890-4560-19cc-986a-56b900117c28-ea0d7a57-d319-4827-833e-ce94bfde1041.swap

[#7 · Accepted solution]

I finally figured it out. Nifi node was unable to talk to cluster because of entry in hosts file. It was looking for partial hostname not fully qualified domain name.

[#8]

Hi @Shishir Saxena,

I have similar issue.

In my /etc/hosts file,

10.244.6.45 nifi-1.nifi-service.nifi-cluster.svc.cluster.local nifi-1

If I use "nifi-1" as hostname in nifi.properties, is it OK?

[#9]

Hello Shishir,

Would you mind to please how do we migrate a standalone Nifi setup to cluster mode?

Thanks

snm1523