Support Questions
Find answers, ask questions, and share your expertise

Migrating HDF 2.0 node from standalone to cluster mode gives error.

I have a HDF 2.0 node that was running in standalone mode. When I convert it to cluster mode, by changing nifi.properties file and restarting HDF 2.0, I am getting following error message.

Cluster is still in the process of voting on the appropriate Data Flow.

I have removed existing flow.xml.gz from conf directory and working with an empty flow file.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Migrating HDF 2.0 node from standalone to cluster mode gives error.

I finally figured it out. Nifi node was unable to talk to cluster because of entry in hosts file. It was looking for partial hostname not fully qualified domain name.

View solution in original post

8 REPLIES 8

Re: Migrating HDF 2.0 node from standalone to cluster mode gives error.

Expert Contributor

is zookeeper running?

Re: Migrating HDF 2.0 node from standalone to cluster mode gives error.

Check in your nifi.properties, there are two properties:

nifi.cluster.flow.election.max.wait.time=5 mins

nifi.cluster.flow.election.max.candidates=

It is probably waiting 5 mins based on the first property. Usually you want to set the second property to a number <= to the nodes in your cluster, and then they will vote on the flow which will likely happen way faster than the 5 mins.

Re: Migrating HDF 2.0 node from standalone to cluster mode gives error.

Expert Contributor

My HDF just gave up suddenly. After restarting NiFi and later the zookeeper node, it gives the error "Invalid State. Cluster is still in the process of voting on the appropriate Data Flow." I have checked the properties suggested by @Bryan Bende, they are set as nifi.cluster.flow.election.max.candidates=1, nifi.cluster.flow.election.max.wait.time=5 mins. @Shishir Saxena I have also checked my /etc/hosts file and it's properly setup. Is there anything i'm not getting right?

Re: Migrating HDF 2.0 node from standalone to cluster mode gives error.

How many NiFi nodes are there? Can you see what is in /var/log/nifi/nifi-app.log for each of the NiFi nodes when this is happening?

Re: Migrating HDF 2.0 node from standalone to cluster mode gives error.

Expert Contributor

There's only one NiFi node @Bryan Bende. The nifi-app log does not suggest any error. I also notice that my flowfile-repo is filling up so fast.

2016-12-22 08:22:20,574 INFO [Timer-Driven Process Thread-13] o.a.n.controller.FileSystemSwapManager Successfully swapped out 10000 FlowFiles from FlowFileQueue[id=c5633890-4560-19cc-986a-56b900117c28] to Swap File /apps1/flowfile-repo/swap/1482387740171-c5633890-4560-19cc-986a-56b900117c28-4c2f21a9-23c1-494a-b92f-89b9a85c4f32.swap
2016-12-22 08:22:20,658 INFO [Timer-Driven Process Thread-13] o.a.n.c.r.WriteAheadFlowFileRepository Successfully swapped out 10000 FlowFiles from FlowFileQueue[id=c5633890-4560-19cc-986a-56b900117c28] to Swap File /apps1/flowfile-repo/swap/1482387740171-c5633890-4560-19cc-986a-56b900117c28-4c2f21a9-23c1-494a-b92f-89b9a85c4f32.swap
2016-12-22 08:22:24,803 INFO [Timer-Driven Process Thread-10] o.a.n.c.r.WriteAheadFlowFileRepository Repository updated to reflect that 10000 FlowFiles were swapped in to FlowFileQueue[id=c5633890-4560-19cc-986a-56b900117c28]
2016-12-22 08:22:25,647 INFO [Timer-Driven Process Thread-3] o.a.n.controller.FileSystemSwapManager Successfully swapped out 10000 FlowFiles from FlowFileQueue[id=c5633890-4560-19cc-986a-56b900117c28] to Swap File /apps1/flowfile-repo/swap/1482387745230-c5633890-4560-19cc-986a-56b900117c28-ea0d7a57-d319-4827-833e-ce94bfde1041.swap
2016-12-22 08:22:25,685 INFO [Timer-Driven Process Thread-3] o.a.n.c.r.WriteAheadFlowFileRepository Successfully swapped out 10000 FlowFiles from FlowFileQueue[id=c5633890-4560-19cc-986a-56b900117c28] to Swap File /apps1/flowfile-repo/swap/1482387745230-c5633890-4560-19cc-986a-56b900117c28-ea0d7a57-d319-4827-833e-ce94bfde1041.swap

Re: Migrating HDF 2.0 node from standalone to cluster mode gives error.

I finally figured it out. Nifi node was unable to talk to cluster because of entry in hosts file. It was looking for partial hostname not fully qualified domain name.

View solution in original post

Re: Migrating HDF 2.0 node from standalone to cluster mode gives error.

Contributor

Hi @Shishir Saxena,

I have similar issue.

In my /etc/hosts file,

10.244.6.45 nifi-1.nifi-service.nifi-cluster.svc.cluster.local nifi-1

If I use "nifi-1" as hostname in nifi.properties, is it OK?

Re: Migrating HDF 2.0 node from standalone to cluster mode gives error.

Rising Star

Hello Shishir,

 

Would you mind to please how do we migrate a standalone Nifi setup to cluster mode?

 

Thanks

snm1523