Titanium ⭐️⭐️⭐️

HDF 2.x - Adding a new NiFi Node to an existing secured NiFi cluster via Ambari

October 5, 2026 at 9:20 AM
3 replies
8437 views
Translate All

With HDF 2.x, Ambari can be used to deploy a NiFi cluster. Lets say you deployed a 2 node cluster and want to go back at a later time and add an additional NiFi node to the cluster. While the process is very straight forward when your NiFi cluster has been setup non-secure (http), the same is not true if your existing NiFi cluster has been secured (https).

Below you will see an existing 2 node secured NiFi cluster that was installed via Ambari:

STEP 1: Add new host through Ambari. You can skip this step if the host you want to install the additional NiFi node on is already managed by your Ambari.

STEP 2: Under "Hosts" in Ambari click on the host form the list where you want to install the new NiFi node.

The NiFi component will be in a "stopped" state after it is installed on this new host.

*** DO NOT START NIFI YET ON NEW HOST OR IT WILL FAIL TO JOIN CLUSTER. ***

STEP 3: (This step only applies if NiFi's file based authorizer is being used) Before starting this new node we need to clear out some NiFi Configs. This step is necessary because of how the NiFi application starts. When NiFi starts it looks for the existence of a users.xml and authorizations.xml files. If they do not exist, it uses the configured "Initial Admin Identity" and "Node identities (1,2,3, etc...)" to build the users.xml and authorizations.xml files. This causes a problem because your existing clusters users.xml and authorizations.xml files likely contain many more entires by now. Any mismatches in these files will prevent a node from being able to join the cluster. If these configurations are not present, the new node will grab them from the cluster it joins. Below shows what configs need to be cleared in NiFi:

*Note: Another option is to simply copy the users.xml and authorizations.xml files from an existing cluster node to the new node before starting the new node.

STEP 4: (Do this step if using Ambari metrics) When a new node is added by Ambari and Ambari metrics are also enabled, Ambari will create a flow.xml.gz file that contains just the ambari reporting task. Later when this node tries to join the cluster, the flow.xml.gz files between this new node and the cluster will not match. This mis-match will trigger the new node to fail to join cluster and shut back down. In order to avoid this problem the flow.xml.gz file must be copied from one of the cluster's existing nodes to this new node.

STEP 5: Start NiFi on this new node. After the node has started, it should successfully join your existing cluster. If it fails, the nifi-app.log will explain why, but will likely be related to one of the above configs not being cleared out causing the users.xml and authorizations.xml files to get generated rather then inherited from the cluster. If that is the case you will need to fix the configs and delete those files manually before restarting the node again.

STEP 6: While you cluster is now up and running with the additional node, but you will notice you cannot open the UI of that new node without getting an untrusted proxy error screen.

You will however still be able to access your other two node's UIs. So we need to authorize this new node in your cluster.

A. If NiFi handles your authorizations, follow this procedure:

1. Log in to the UI of one of the original cluster nodes.

The "proxy user requests" access policies is needed to allow users to access the UI of your nodes.

NOTE: There may be additional component level access policies (such as "view the data" and "modify the data") you may also want to authorize this new node for.

B. If Ranger handles your NiFI authorizations, follow this procedure:

1. Access the Ranger UI:

2. Click Save to create this new user for your new node. Username MUST match exactly with the DN displayed in the untrusted proxy error screen.

3. Access the NiFi service Manager in Ranger and authorize your new node to your existing access policies as needed:

You should now have a full functional new node added to your pre-existing secured NiFi cluster that was deployed/installed via Ambari.

bpreachuk

Gold ⭐️

Additional information from @Matt Clarke:

There are three files crucial to a new node being able to successfully join an existing cluster....(flow.xml.gz, users.xml, and authorizations.xml). All three of these files (flow, users, authorizations) must match before a node will be allowed to join an existing cluster.

The flow.xml.gz file contains everything you have added while interfacing with the UI.

All nodes must have matching flow.xml.gz files in order to join cluster. All you need to do is copy flow.xml.gz file from the original cluster node to the new node, make sure ownership is correct, and restart new node.

Normally these files will be given out by the cluster to any new node who has none of them; however, if Ambari metrics are enabled and a flow.xml.gz does not exist, Ambari generates a flow.xml.gz file that contains only the Ambari reporting task. Because of this the new node will not match and will be unable to join the cluster. A NiFi cluster will never overwrite an existing flow.xml.gz file on a new node with its own.

Secured NiFi clusters also requires that the users.xml and authorizations.xml file match if file based authorization is used. The users and authorizations XML files only come in to play when NiFi is secured and using the local file based authorization. If secured, the only time a cluster will hand out the users and authorizations XML files is if they don't exist as well.

Bottom line... If you add a new NiFi host via ambari, it will try to join cluster. If it fails and shuts back down, copy the above the files from one of the existing nodes to the new node and restart via Ambari.

Translate