Member since
07-30-2019
3397
Posts
1619
Kudos Received
1001
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 477 | 11-05-2025 11:01 AM | |
| 364 | 11-05-2025 08:01 AM | |
| 564 | 11-04-2025 10:16 AM | |
| 723 | 10-20-2025 06:29 AM | |
| 863 | 10-10-2025 08:03 AM |
11-15-2016
08:22 PM
1 Kudo
@sagar gaikwad Difference between templating the entire NiFi canvas and copying the actual flow.xml.* files. When you generate a template it is saved as and xml and contains all the configuration from the canvas with the EXCEPTION of sensitive property values (passwords). Sensitive processor properties are encrypted using the sensitive props key configured in the nifi.properties file. So by removing these values when a template is created the templates to be loaded into any other NiFi successfully. If you capture the flow.xml.* files directly from NiFi, you will also need to save/reserve the value set for the sensitive props key in order to use them on another system. The above is true no matter which version of NiFi you are using. NiFi 0.x /HDF 1.x versions: With these versions of NiFi you have a NiFi Cluster Manager (NCM) and NiFi nodes. The Nodes all have identical flow.xml.gz files and an identical list of templates inside the /templates directory. The NCM has a flow.tar file. This file is not the same as what is on the nodes. The flow.tar consists of numerous files important to the NCM only. (NCM specific controller services, primary node designator, etc..) The flow.tar also includes a flow.xml file. It is from this flow.xml file on the NCM that the flow.xml.gz and templates inside the /templates dir on the nodes are derived from. You can build a new cluster if you have the flow.tar file as it contains everything new nodes need plus what ta new NCM needs. The flow.tar cannot be used on a standalone install of NiFi nor can it be used by the nodes directly themselves. It is intended for an NCM only. The flow.xml.gz file from any of the nodes can be used to stand-up a standalone NiFi instance. You may choose to keep a backup of both. NiFi 1.x /HDF 2.x versions: These versions of NiFi have eliminated the need for a NCM (zero-master clustering). As a result every instance of Nifi is a node and they all have identical flow.xml.gz files. These flow.xml.gz files do not contain the exact same content as the nodes from 0.x versions of NiFi. The templates have all been moved into the flow.xml.gz file and there is no longer a need for a separate /templates dir. With this version you can keep a backup or flow.xml.gz from any node. The flow.xml.gz file can be used to buidl another cluster or a standalone NiFi instance. Thanks, Matt
... View more
11-15-2016
02:09 PM
@Iyappan Gopalakrishnan Adding new users to a secured NiFI 0.x / HDF 1.x version of NiFi can be done directly via the NIFi UI. I am assuming you have already secured your NiFi and manually added the initial admin to the authorized-users.xml file. Adding new users is pretty easy. Simply have the new user attempt to access the https web address for your nifi 0.7 install. After authenticating successfully (Done via client certs, ldap, etc), they will see a screen telling them they are not authorized for access and giving them the opportunity to request access. Once they request access, the "Admin" user who does have access to the UI will need need to go into the users UI by clicking on the below Icon which can be found in the upper right corner of the NiFi UI: From within that UI you will see all existing users and any pending authorization users. Simply click on the pencil icon to the right of any pending users to authorize them for 1 or more of the available user roles (Administrator, Dataflow Manger, Read Only, or Provenance). The administration-guide included with your install can give you more details on each of the roles and what privileges each grants to the users. Thanks, Matt
... View more
11-15-2016
01:53 PM
@Rajkumar Singh @Iyappan Gopalakrishnan The link here references an administration-guide for NIFI 1.x baseline. The NIFI 0.x baseline is very different. I suggest reading the administration-guide that was bundled with your download version for more accurate information on adding and setting up new users. Thanks, Matt
... View more
11-15-2016
01:45 PM
@Karthik Narayanan The link you reference is good for troubleshooting adding new nodes in NiFi 0.x and HDF 1.x versions of NiFi which had an NCM. NiFi 1.x and HDF 2.x version are very different now and do not have an NCM.
... View more
11-15-2016
01:43 PM
2 Kudos
@mayki wogno The reported error message is a little misleading, but the key to your issue is reported in the "Caused by" portion of the stack trace: Causedby: org.apache.nifi.controller.UninheritableFlowException:ProposedAuthorizerisnot inheritable by the flow controller because of Authorizer differences:ProposedAuthorizationsdonot match current Authorizations Verify that the authorizers.xml file on your new node matches the configuration on your other existing connected nodes. Remove the users.xml and authorizations.xml files from your new node as well be restarting. The new Node should be getting these files from the cluster when it joins. If you continue to see the same error message on restart, try copying the users.xml and authorizations.xml files from one of your existing connected nodes to this new node and restart again. Thanks, Matt
... View more
11-15-2016
01:22 PM
1 Kudo
@Saikrishna Tarapareddy S2S will not use LDAP for authentication. It uses the DN from the client side cert you created/obtained. If the Identity mapping properties @Koji Kamimura mentioned above are configured on the receiving side NiFi, they will be applied against that client side certificate DN that is presented. The resulting mapped value is what will need to be authorized to access S2S details and any input ports you wish to post to. If the identity mapping stuff is not configured, the full DN will need to authorized. Thanks, Matt
... View more
11-14-2016
09:54 PM
Only the NiFi 0.x or HDF 1.x versions of NiFi use a NCM. NiFI 1.x or HDF 2.x versions have moved to zero master clustering and do not have an NCM anymore (HA control plane). The routing of data you are referring to is specific to data being sent to your NiFi cluster via Site-to-Site (S2S). S2S does make sure that data continues to route to only the available destination nodes. Matt
... View more
11-14-2016
09:49 PM
I believe the process you have is spot on and keeps the number of processors to a minimum. Matt
... View more
11-14-2016
09:47 PM
1 Kudo
@ambud.sharma Each Node in a NiFi cluster runs its own copy of the dataflow and works on its own set of FlowFiles. Node A for example is unaware of the existence of Node B. NiFi does persist all FlowFiles (attributes and content) in to local repositories on each node in the cluster. That is why is is important to make these repo fault tolerant (For example using RAID 10 Disk for your repos). Should a node go down, as long as you have access to those repos and copy of the flow.xml.gz, you can recover your dataflow where it left off, even if that means spinning up a new NiFi and pointing it at those existing repos. NiFi comes with no automated built in process for this. While Nodes at this current time are not aware of other nodes or the data the currently have queued, This is a roadmap item for a future version of NiFi. At this time the HA Data plane stuff has not been committed to any particular release to the best of my knowledge. Thanks, Matt
... View more
11-14-2016
09:38 PM
1 Kudo
@Saikrishna Tarapareddy
S2S does not use LDAP for server authentication. S2S uses the keystore and truststore provided in the nifi.properties file to establish a secured mutual authenticated connection between two secured NiFi instances/clusters. The destination NiFi dictate whether the S2S connection will be secure or not. If you have secured your destination NiFi, then only a source NiFi (one with the RPG) that has been configured with its own server keystore and truststore will be able to connect. Since S2S relies on certificates for mutual authentication. The user authentication you choose to use can be different on each NiFi installation. LDAP on one, certs on another, etc... Thanks, Matt
... View more