Nifi: What are some best practices to support a multi-tenant Team>Project heirarchy through access policies?

[#1]

I have a secured nifi cluster that is identifying users via LDAP.

I wish to configure a security policy whereby multiple teams can use nifi for their multiple projects without risk of modifying or viewing other team's flows.

I understand that one can use ranger, or nifi's built-in authorization scheme, are there [dis]advantages to either approach?

The natural (naïve?) layout seems to be a process group for each team, and then each team may have multiple process groups underneath their team's process group representing projects. Are there any gotchas to be aware of with this setup?

One perceived gotcha is that this layout seems to clash with the ListSFTP->RPG->FetchSFTP pattern. A subgroup cannot directly receive RPG inputs, so anytime the List->RPG->Fetch pattern is used, an input port must be created at the root flow and then mapped all the way back into the sub process. This is a problem because this pattern will be common, and will require permissions on the root flow to create the input ports. Am I misunderstanding something, and is there a more elegant solution?

[#2]

You seem to have a good understanding of the situation...

There isn't really a big pro or con with NiFI's authorizer vs Ranger... If you are already using Ranger for other things then it gives you a nice central place to manage all policies across your enterprise and a nice way to look at audit logs. The current downside would be that when using an external authorizer like Ranger, NiFi's UI currently doesn't show anything about the policies, so you have to jump back and forth a bit between two UIs sometimes. This is only temporary though and in a future release there are plans to offer a read-only view of policies on the NiFi side when using an external authorizer. Sometimes it is helpful to start out with NiFi's authorizer to get an understanding of the policies before attempting to set them up in Ranger.

Regarding the RPG pattern, you are correct that this is one unfortunate scenario regarding multi-tenancy... I think the real way that this should be addressed is by creating a more user-friendly way to do load balancing of a connection across the cluster, most likely a special way to connect two components and indicate the connection should transfer data to each node across the cluster. The RPG approach is used simply because that is all that exists, but site-to-site was really meant for transfer between two separate instances.

An alternative option is always to have each team with its own NiFi instance and then they can manage the root canvas themselves, and they can send data across teams via site-to-site, but I realize this requires managing more clusters, just throwing it out there.

[#3]

Thanks for the answer, it does seem like another mechanism besides RPG-to-self is needed to rebalance flowfiles generated on a single node.

[#4 · Accepted solution]

@David Miller

NiFi's default File based authorizer:

Advantages:

- supports user groups (This can make setting up authorizations for team a lot less cumbersome.)

- Integrated within NiFi so no need to worry about connective issues with external service.

Disadvantages:

- There is no way currently to sync the user with LDAP. User must be added manually.

Ranger based authorizer:

Advantages:

- Ranger can be setup to sync users from LDAP.

- Authorizing new users does not require authorization admin to have access to NiFi's UI.

Disadvantages:

- Ranger user groups are not supported yet. (Each and every user must be added to any policy required)

Here is a helpful link that maps Ranger Policies to NiFi's Default user authorizations:

https://community.hortonworks.com/content/kbentry/115770/nifi-ranger-based-policy-descriptions.html

-----

You are correct that the most common approach to user/team managed authorization is through the user of unique process groups added to the root canvas level. Sub-process groups by default inherit their access policies from the parent process group.

The only thing to be aware of is the use of NiFi's Site-To-Site (S2S)capability. Site-To-Site Remote input and output ports must be added at the root canvas level. So when it comes to using S2S to receive or send data from a NiFi, you would need a admin level user who has the ability to add these components to to the root canvas level for your users and connect them to a process group(s) that your users/teams are authorized for. The other side of a S2S connection is a Remote Process Group (RPG). These RPGs can be added at any level (sub-process group) in a dataflow so special considerations are needed here.

A typical approach might be to create a Remote Input port for each team (Process group) and connect that port(s) to their assigned process group. Once in the team group, a routing processor could be shared by all sub teams/users so direct a particular feed of incoming S2S data to a particular sub-process group. Teams are still going to need to work with admins to authorize remote NiFi instance to connect to these ports, so it cannot be completely team managed after creation.

Thanks,

Matt

[#5]

Thanks Matt, the pre-provisioned input port idea does seem workable.