Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NiFi Clustering: One NCM to manage multiple separate "logical" Nifi clusters

avatar
Expert Contributor

Is it possible to configure a NCM to manage multiple distinct sets of slave (nodes)? For example could one NCM coordinate the following scenario:

  • Flow1 deployed to Nodes 1-3 by NodeControlManager 1
  • Flow2 deployed to Nodes 4-6 by NodeControlManager 2

Does NCM have the ability to selectively deploy flows to separate nodes, effectively creating separate logical sets of NiFi cluster workers (nodes)?

In summary can we have ... "One NCM to rule them all and in the darkness bind them"

1 ACCEPTED SOLUTION

avatar

No, and not sure you should.

2 NCMs managing 2 clusters (nodes 1-3 and 4-6) - perfectly ok. NCM (NiFi Cluster Manager) is a lightweight process and can be colocated with any of the processing nodes.

You can set up a site-2-site channel to have 2 clusters communicate over a centralized data plane, too.

View solution in original post

4 REPLIES 4

avatar

No, and not sure you should.

2 NCMs managing 2 clusters (nodes 1-3 and 4-6) - perfectly ok. NCM (NiFi Cluster Manager) is a lightweight process and can be colocated with any of the processing nodes.

You can set up a site-2-site channel to have 2 clusters communicate over a centralized data plane, too.

avatar
Expert Contributor

Thanks Andrew. As an alternative, would it be safe to deploy to running instances of NCM on a single server? Perhaps binding them to separate ports? Or conflicts could arise?

avatar

Yes, you could do that, as long as ports don't conflict.

avatar
Expert Contributor