Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NIFI: Node discovering for Cluster

Solved Go to solution

NIFI: Node discovering for Cluster

Explorer

Hi,

For example, I have 3 NiFi nodes connected to one cluster.

If I'll have additional 2 NiFi nodes, is there a way to "ask them to join" cluster?

All nodes in one private network.

Thank you for your help.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NIFI: Node discovering for Cluster

@Gu Gur

If you configure the two nodes the same way as the current three nodes in the cluster, they should "ask to join" automatically.

Which version of NiFi are you using?

View solution in original post

8 REPLIES 8
Highlighted

Re: NIFI: Node discovering for Cluster

@Gu Gur

If you configure the two nodes the same way as the current three nodes in the cluster, they should "ask to join" automatically.

Which version of NiFi are you using?

View solution in original post

Highlighted

Re: NIFI: Node discovering for Cluster

Explorer

1.0.1

but how will they find master node?

it means that I need to reconfigure all 5 nifi nodes in order to work?

Highlighted

Re: NIFI: Node discovering for Cluster

@Gu Gur

No, you only have to configure the two new nodes to be part of the cluster that is already running. Make sure to configure to use the same root node in zookeeper, set this property nifi.zookeeper.root.node, to the same values that the other three nodes are using, and of course configure the cluster node properties.

Highlighted

Re: NIFI: Node discovering for Cluster

Explorer

Hi, Thank you very much for answer. So can I implement it using ExecuteScript processor ? Could you please give some tips where to look at.

Highlighted

Re: NIFI: Node discovering for Cluster

@Gu Gur

You could you the ExecuteScript processor, but that would probably be more work than just doing it manually since it is only two nodes you are adding.

I am assuming you have not secured the cluster.

You could just copy the conf directory over to the new nodes, and then update the all entries to match the name of the new host.

For example, assuming the current nodes are node1.localdomain, node2.localdomain and node3.localdomain, then copy node3's conf directory to a new node you want to add, you could just change everywhere you have node3 to node4, or whatever is the new node system name.

Are you following me?

Highlighted

Re: NIFI: Node discovering for Cluster

@Gu Gur

I agree, that would be a nice tool. But a difficult task because there are unique environments for almost each user and multiple environments for each user.

Highlighted

Re: NIFI: Node discovering for Cluster

Master Guru

@Gu Gur

You just need to make sure the following lines in the new nodes nifi.properties file match the same lines in the nifi.properties files of those nodes already in your cluster.

I also recommend setting/updating the following line in the nifi.properties file on every node in your cluster each time you add or remove a node:

nifi.cluster.flow.election.max.candidates=

It is blank by default which means the UI will be unavailable for 5 minutes whenever the cluster is restarted while NiFi waits for the leader election to complete.

Of course all your rest of the "cluster node properties" must be set on your new nodes as well.

Any changes to the nifi.properties file will require a NiFi restart to take affect

Thanks,

Matt

Highlighted

Re: NIFI: Node discovering for Cluster

Explorer

Yes, I follow you ) But it's just example 2 more new nodes. It could be 10-15 and so on...

So in this case it would be cool to automate this process somehow.

Don't have an account?
Coming from Hortonworks? Activate your account here