@Andre Labbe 1. NiFi is designed to prevent changes while a node is disconnected. Each node runs its own copy of the flow.xml.gz. When a node is disconnected, the cluster coordinator has not means to determine the state of components on that disconnected node. If changes were allowed to be made, that node would be unable to automatically rejoin the cluster without manual intervention to bring them back in sync. To regain control of the canvas you can access the cluster UI and drop the disconnected node from the cluster. Keep in mind that if you want to later rejoin that node, you will need to make sure the flow.xml.gz, users.xml (if used when NiFi is secured), and authorizations.xml (if used when NiFi is secured) are all in-sync with what is currently be used in cluster (you can copy these from other active cluster node). *** Be mindful that if a change you made removed a connection that contained data on your disconnected node, that data will be lost on startup once you replace the flow.xml.gz. If Nifi cannot find the connection to place the queued data it is lost. 2. If you set a Processor component to execute on "Primary node", it dose not run. What component and how do you know it did not run? If have not see this happen before. Processors that use no cluster friendly protocols should be run on Primary Node only to prevent data duplication as you noted above. If you are consuming a lot of data using one of these protocols, ti is suggested you use the List/Fetch (example: listSFTP/FetchSFTP) processors along with NiFi's Site-To-Site (S2S) capability to redistribute the listed FlowFiles to all nodes in your cluster before the Fetch. Also note that you need an odd number of Zookeeper nodes in order to have Quorum. (3 minimum). Using the embedded zookeeper means you lose that ZK node anytime you restart that NiFi instance. I don't recommend using embedded ZK in production. Thanks, Matt ... View more

@pawan soni Is the UI of the NiFi instance running on Node 1 reachable via port 9090? The RPG reports some communication issues there. This may have just been the result of node 1 restart? The invalid state message indicates the state of your Remote Process Group is Enabled ( ); however, the Remote Input Port is stopped (thus an invalid state for data transfer). Either start the "From File" input port or "disable that port in your RPG to get rid of this ERROR. Thanks, Matt ... View more

@David Miller NiFi's default File based authorizer: Advantages: - supports user groups (This can make setting up authorizations for team a lot less cumbersome.) - Integrated within NiFi so no need to worry about connective issues with external service. Disadvantages: - There is no way currently to sync the user with LDAP. User must be added manually. Ranger based authorizer: Advantages: - Ranger can be setup to sync users from LDAP. - Authorizing new users does not require authorization admin to have access to NiFi's UI. Disadvantages: - Ranger user groups are not supported yet. (Each and every user must be added to any policy required) Here is a helpful link that maps Ranger Policies to NiFi's Default user authorizations: https://community.hortonworks.com/content/kbentry/115770/nifi-ranger-based-policy-descriptions.html ----- You are correct that the most common approach to user/team managed authorization is through the user of unique process groups added to the root canvas level. Sub-process groups by default inherit their access policies from the parent process group. The only thing to be aware of is the use of NiFi's Site-To-Site (S2S)capability. Site-To-Site Remote input and output ports must be added at the root canvas level. So when it comes to using S2S to receive or send data from a NiFi, you would need a admin level user who has the ability to add these components to to the root canvas level for your users and connect them to a process group(s) that your users/teams are authorized for. The other side of a S2S connection is a Remote Process Group (RPG). These RPGs can be added at any level (sub-process group) in a dataflow so special considerations are needed here. A typical approach might be to create a Remote Input port for each team (Process group) and connect that port(s) to their assigned process group. Once in the team group, a routing processor could be shared by all sub teams/users so direct a particular feed of incoming S2S data to a particular sub-process group. Teams are still going to need to work with admins to authorize remote NiFi instance to connect to these ports, so it cannot be completely team managed after creation. Thanks, Matt ... View more