Nifi Cluster- Proxy hostname

[#1]

Hello,

I was extending my HDF cluster (Kerberos -Active Directory)/ Ranger / SSL enabled ) by adding one node using Ambari and I came accross the following behavior I'm struggling to explain & address.

The nodes of my cluster all have a AD principal CN=nodename, OU=some OU, DC=dc1, DC=dc2.

All nodes are members of a group "NifiClusterMember"

Ranger is in sync, and the required policies are setup to allow R/W access to Proxy for NifiClusterMember.

While trying to log on to the new node, I received the error "Untrusted Proxy - <Nodename>". In this case, the "short" name. not the FDQN.

I had to create <short name> as in internal user in Ranger to setup the access.

While it's currently running OK, i'd like to clean up the situation the way it should. However I could not find what part of the configuration is driving the behavior I encounter.

I take any tips 🙂

thanks

[#2 · Accepted solution]

@Christophe Vico

In order for users or servers to access any secured NiFi end-point a series of steps occur:

1. User/server authentication: For users this sounds like it is being done via AD which you have configured via Ambari (this sets up the login-identity-providers.xml file in NiFi). For servers this is done via server certificates that are configured for each of your NiFi nodes. If and only if user/server authentication is successful, then the process moves on to step two. Do a verbose listing of the keystore on your newly added node to verify DN structure.

2. User/server Pattern mapping: NiFi can be configured to use pattern mappings to extract a portion of the return DN or principal before passing the resulting mapped value to the configured authorizer (in your case the authorizer is Ranger). These mappings can be setup in Ambari which will update the nifi.properties file.

by default, pattern mapping is not enabled. All Authenticated entities no matter how the authentication was completed are passed through every configured mapping looking for a match. If no matches are found, the full return is sent to the authorizer. Check to see if any mappings have been setup that are matching on your new server's DN.

3. User/Server Authorization: User/server authorization is check to see what NiFi resources a particular authenticated entity has been granted access to.

Note: You can follow the authentication and authorization process by looking in the nifi-user.log.

Thanks,

Matt

[#3]

@Matt Clarke,

I indeed have mapping set:

nifi.security.identity.mapping.pattern.dn=^cn=(.*?),ou=(.*?),ou=(.*?),ou=(.*?),dc=(.*?),dc=(.*?)$ nifi.security.identity.mapping.pattern.dn2=^CN=(.*?), OU=(.*?), OU=(.*?), OU=(.*?), DC=(.*?), DC=(.*?)$ nifi.security.identity.mapping.pattern.kerb= nifi.security.identity.mapping.value.dn=$1 nifi.security.identity.mapping.value.dn2=$1 nifi.security.identity.mapping.value.kerb=

I'm not too sure how to interpret these (i'm new to this actually)

thanks!

[#4]

@Christophe Vico

Based on the DN pattern you provided in your question (CN=nodename, OU=some OU, DC=dc1, DC=dc2). None of the mapping you shared should have matched. Are you sure there isn't a third OU in there? The closest mapping is

nifi.security.identity.mapping.pattern.dn2=^CN=(.*?), OU=(.*?), OU=(.*?), OU=(.*?), DC=(.*?), DC=(.*?)$

nifi.security.identity.mapping.value.dn2=$1

The mapping pattern above takes the DN that starts with CN=<capture group 1>, OU=<capture group 2>, OU=<capture group 3>, OU=<capture group 4>, DC=<capture group 5>, DC=<capture group 6>

each "(.*?)" creates a java capture group of everything between "=" and ",".

It then returns the value of <capture group 1> only as dictated by the "$1". It is this resulting value that is passed to teh authorizer.

assuming there is a third , OU= in your servers DN, this would explain exactly what you are seeing.

Thanks,

Matt

[#5]

@Matt Clarke,

Looking at the AD object, I see

CN=<name>,OU=<OU1>,OU=<OU2>,OU=<OU3>,DC=<DC1>,DC=<DC2>

Which would make my mapping correct, am i right ?

Thanks

Christophe

[#6]

@Christophe Vico

Exactly, so only <name> as you saw would have been passed to ranger for authorization. It is working correctly based upon your current configuration.

Can you access the UI of all of the other nodes without issue? Do there DNs match the same patterns on those nodes.

If my above answer addressed your question, please accept that answer.

Thank you,

Matt

[#7]

Hi @Matt Clarke,

the 3 nodes principals are located in the same OU, and have the same patterns (servicePrincipalName does include the full FQDN)

I can access the 3 nodes by their full FQDN, host 1 & 2 report to ranger as their FDQN, host 3 only by its short name.

I'm still struggling to understand why i have different behavior for host 3

Thanks

[#8]

@Christophe Vico

Confirm the DN of the PrivateKeyEntry in the keystore of all you nodes.

# keystore -v --list -keystore <keystorename>

Also confirm the patterns have been set on all nodes in the nifi.properties file.

All it takes is a slight difference (extra space, missing space, case sensitivity, etc...)

Thanks,

Matt

[#9]

Hi @Matt Clarke

You nailed it again.

The two "old" hosts have Owner : CN=FQDN, etc

The new host has Owner: CN=<Shortname>, etc

Seems the 2 initial hosts cert have been created by a different script that the one I got.

It's not clear where is the problem. I'll check and update the cert where needed.

Thanks!