NiFi Clustering: One NCM to manage multiple separate "logical" Nifi clusters

[#1]

Is it possible to configure a NCM to manage multiple distinct sets of slave (nodes)? For example could one NCM coordinate the following scenario:

• Flow1 deployed to Nodes 1-3 by NodeControlManager 1
• Flow2 deployed to Nodes 4-6 by NodeControlManager 2

Does NCM have the ability to selectively deploy flows to separate nodes, effectively creating separate logical sets of NiFi cluster workers (nodes)?

In summary can we have ... "One NCM to rule them all and in the darkness bind them"

[#2 · Accepted solution]

No, and not sure you should.

2 NCMs managing 2 clusters (nodes 1-3 and 4-6) - perfectly ok. NCM (NiFi Cluster Manager) is a lightweight process and can be colocated with any of the processing nodes.

You can set up a site-2-site channel to have 2 clusters communicate over a centralized data plane, too.

[#3]

Thanks Andrew. As an alternative, would it be safe to deploy to running instances of NCM on a single server? Perhaps binding them to separate ports? Or conflicts could arise?

[#4]

Yes, you could do that, as long as ports don't conflict.

[#5]

Found additional information in the article by @mclark here: https://community.hortonworks.com/articles/8607/how-to-create-nifi-fault-tolerance-using-multiple.ht...