Setting Up a Secure NiFi Registry Instance

[#1]

I have a 3 node NiFi cluster set up and running in our environment.  I used a user called "ec2-user" to perform the NiFi install. For this initial install (in a development environment) I used the NiFi Toolkit to create the certificates for the 3 nodes as well as for the ec2-user

I would now like to use the NiFi registry tool to create additional NiFi users.

I have been following this post by alim: Setting Up a Secure Apache NiFi Registry - Cloudera Community - 247753.

I started with "Registry Configuration" in this post since I had already used the Toolkit to generate the keystore, truststore and a client certificate for the ec2-user.

I copied the keystore and truststore to the conf directory of the Registry install.

I then copied the values rom the keystore and truststore properties from the nifi.properties file into the corresponding values in the nifi-registry.properties file.

I also modified the HTTP and HTTPS web properties as indicated in the post.

I then modified the authorizers.xml file.  First in the userGroupProvider section, adding the "ec2-user" DN to the initial Admin Identity 1" property. And, then in the accessPolicyProvider section, adding the "ec2-user" DN to the "Initial Admin Identity" property.

I then copied the certificate associated with the ec2-user to the nifi1 host browser.

I then started the Registry ./bin/nifi-registry.sh start

I then accessed the url:  (I changed the host to nifi1 instead of localhost):  https://nifi1:18443/nifi-registry .

I was prompted for the ec2-user certificate which i provided.

Then I was able to access the nifi-registry GUI but i do not see the wrench on the far upper right of the page.

Also, when I look at the nifi-registry log, I see that the Kerberos service ticket not supported by the NiFi Registry.  Also receive an AccessDeniedExceptionMapper:  identity CN=ec2-user does not have permission to access the requested resource.

Can you provide guidance on how I can create a user with administrator access in the NiFi Registry tool?  Are there any previous tickets that describe how to to do this?

Thank you for any guidance, recommendations....

[#2]

@davehkd 
Sounds like your NiFi-Registry service install went well.  There is not such thing as a NiFi registry tool. I assume you were referring to the NiFi TLS toolkit you used to generate your certificates and keystores?

You are dealing with an issue related to your initial admin.

1. When you configured the user-group-provider and file-access-policy provider, you stated that you used your "ec2-user" DN; however, the authorization exception you shared is as follows"

AccessDeniedExceptionMapper:  identity CN=ec2-user does not have permission to access the requested resource.

That is not a the user's full DN in that exception.  To NiFi and NiFi-Registry the user identity string "CN=ec2-user. OU=NiFi" is not the same user as "CN=ec2-user".  What appears to me from your description is the full user DN has been created in the users.xml and authorization for that full DN setup in the authorizations.xml.  

So to answer why your "CN=ec2-user, OU=NiFi" DN resulted in NiFi-Registry seeing your user identity as only "CN=ec2-user" comes down to identity mapping properties. So in yoou nifi-registry.properties file you'll want to look for sets of properties like the following:

nifi.registry.security.identity.mapping.pattern.<some string>=<some regex pattern with 1 or more capture groups>
nifi.registry.security.identity.mapping.value.<some string>=$1
nifi.registry.security.identity.mapping.transform.<some string>=NONE

My guess here is you will find some regex pattern that matches on your user's full DN and the corresponding value property is then only returning "CN=ec2-user".  

You can remove or modify the matching mapping pattern or change your initial admin to just "CN=ec2-user" instead of using full DN.  Either will work.  Keep in mind if you decide to go with "CN=ec2-user", then you will need edit the authorizers.xml to use "CN=ec2-user" instead of full DN.   You will also need to remove the current users.xml and authorizations.xml files created by the authorizer on first startup so they get recreated with these new identity strings.  The authorizer providers only generate the authorizations.xml and users.xml on startup if they do NOT already exist.

2. the following output you reported from your nifi-registry-app.log is normal and expected:

Kerberos service ticket not supported by the NiFi Registry.

This is not an ERROR log.  It is simply reporting that the spnego related kerberos properties in the nifi-registry.properties file were not configured and thus kerberos authentication via spnego is not supported.  You can just ignore this notification.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

[#3]

Hello Matt, thanks for the response.

I checked the nifi-registry.properties file, and the properties you suggested I check i.e., 

nifi.registry.security.identity.mapping.pattern.<some string>=<some regex pattern with 1 or more capture groups>
nifi.registry.security.identity.mapping.value.<some string>=$1
nifi.registry.security.identity.mapping.transform.<some string>=NONE

are all commented out.

I first stopped the nifi-registry service by issuing the ./nifi-registry.sh stop command.

I then tried what you had recommended next i.e., I changed the initial admin to just "CN=ec2-user" in the authorizers.xml file.  I then deleted the users.xml and authorizations.xml files. I then restarted the nifi-registry service.

And, lo and behold, when I launched the GUI i.e., https://nifi1:18443/nifi-registry, the wrench appeared in the upper right corner!  

A final question.  If I want to successfully add additional NiFi users for the NiFi cluster I have stood up, do I need to set up an LDAP? Is there a good url/reference you could point me to on how to do this?

Thank you Matt

[#4 · Accepted solution]

@davehkd 
Keep in mind that you are not adding users to either NiFi or NiFi-Registry that can be used for authentication.   You are only adding "user identities" that match the user Identity that is returned post successful authentication. 

For example: Your current method of authentication is via a mutual TLS exchange.  Once you pass your client certificate NiFi uses the DN from that certificate as the "user Identity" which is looked up locally to see which authorizations have been associated with that "user identity" (case sensitive).

You can additional "user identities" manually right now via the NiFi and NiFi-Registry UI.  Then you can assign authorizations to those newly added "user identities".     You can use the NiFi TLS toolkit to also create a certificate for each additional user to use for authentication via the mutual TLS exchange.

Many users of NiFi and NiFi-Registry use alternate methods of user authentication with "ldap-provider" as probably the most common.  If you set up ldap-provider in the login-identity-providers.xml, you would be able to authenticate users via that ldap through a login window presented by NiFi instead of needing to use TLS certificates.  You can then optionally, also add the ldap-user-group-provider to the configuration in the authorizers.xml to sync "user identities" and "group identities" from your ldap server so you do not need to manually add these identities yourself via the UI.

Hope this clarifies some things for you.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

[#5]

Thanks again Matt for the excellent guidance and help!