Support Questions

Find answers, ask questions, and share your expertise

NiFi Clustering: One NCM to manage multiple separate "logical" Nifi clusters

Contributor

Is it possible to configure a NCM to manage multiple distinct sets of slave (nodes)? For example could one NCM coordinate the following scenario:

  • Flow1 deployed to Nodes 1-3 by NodeControlManager 1
  • Flow2 deployed to Nodes 4-6 by NodeControlManager 2

Does NCM have the ability to selectively deploy flows to separate nodes, effectively creating separate logical sets of NiFi cluster workers (nodes)?

In summary can we have ... "One NCM to rule them all and in the darkness bind them"

1 ACCEPTED SOLUTION

No, and not sure you should.

2 NCMs managing 2 clusters (nodes 1-3 and 4-6) - perfectly ok. NCM (NiFi Cluster Manager) is a lightweight process and can be colocated with any of the processing nodes.

You can set up a site-2-site channel to have 2 clusters communicate over a centralized data plane, too.

View solution in original post

4 REPLIES 4

No, and not sure you should.

2 NCMs managing 2 clusters (nodes 1-3 and 4-6) - perfectly ok. NCM (NiFi Cluster Manager) is a lightweight process and can be colocated with any of the processing nodes.

You can set up a site-2-site channel to have 2 clusters communicate over a centralized data plane, too.

Contributor

Thanks Andrew. As an alternative, would it be safe to deploy to running instances of NCM on a single server? Perhaps binding them to separate ports? Or conflicts could arise?

Yes, you could do that, as long as ports don't conflict.

Contributor
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.