Support Questions
Find answers, ask questions, and share your expertise

How to give permissions to users to access Nifi UI through Ranger policy

Hello,

How to give permissions to users to access Nifi UI through Ranger policy?

Scenario of the setup:
1. Versions used - Nifi 1.2.0, Ranger 0.7.0

2. Cluster setup of Nifi through Ambari.

3. Ranger is authenticated with LDAP. Created Ranger policy to access Nifi UI.

4. Nifi UI is accessible through LDAP users.

I wanted to give permissions to users to access Nifi UI. When I'm giving specific LDAP user in a policy I won't able to login Nifi UI. I get below message from Nifi UI.

Not authorized for the requested resource. Contact the system administrator.

From Nifi Logs

NiFi Web Server-8342] o.a.n.w.a.c.AccessDeniedExceptionMapper cn=rverma,ou=People,dc=ex,dc=com does not have permission to access the requested resource. Not authorized for the requested resource. Returning Forbidden response.

Ranger policy page - screenshot attached.

39999-ranger-policy-to-access-users.png

I have tried to add other users but still no success. If I add {user} in 'select user' field all LDAP users get access to Nifi UI.

Please suggest where I can define permissions/restrictions for the users.

Thanks,

Suraj

18 REPLIES 18

Looks like Nifi is sending the user name as "cn=rverma,ou=People,dc=ex,dc=com", but in ranger access is given to user "rverma". Check if you can sync users in ranger with the full name Nifi is using.

You can refer the below articles.

https://community.hortonworks.com/articles/58769/hdf-20-enable-ranger-authorization-for-hdf-compone....

https://community.hortonworks.com/articles/57980/hdf-20-apache-nifi-integration-with-apache-ambarir....

https://issues.apache.org/jira/browse/RANGER-1224

@vperiasamy

Thanks you for the suggestions.

I have gone through the links you've given but still I'm unable to access my Nifi UI through LDAP user. Can you please suggest is there anything that need to do in Ranger policy. As I'm able to access ranger with LDAP user but when same users I give in Ranger policy I'm unable to access NIfi UI.

Thanks

Suraj

Mentor

@suraj l

Have a look at this HCC document

@Geoffrey Shelton Okot @vperiasamy

I have made the required changes as per the given links. How to remove anonymous user by getting default login. I ahve not given anonymous user in ranger policy.

Mentor

@suraj l

have you succeeded in giving your Nifi users UI access through ranger? If so well and good , I don't quite understand the next part of your question. "How to remove the anonymous user by getting default login. I have not given anonymous user in ranger policy."

Can you elaborate

@Geoffrey Shelton Okot

have you succeeded in giving your Nifi users UI access through ranger?

Ans> yes

In Ranger policy, when I give {user} in select user tab I can login with LDAP users. Please check screenshot.

40033-ranger-1.png


Also when I hit the Nifi UI (https://nifihost:9091/nifi) automatically it gets logged in with anonymous user. I do not get login page, but there is option to do login. As per attached screenshot.

40034-nifi-1.png

My question is, I wanted to remove this default anonymous login. Please suggest, where I can do changes in configurations.

Also, In Ranger Policy if I remove {user} from select user tab and give specific LDAP users then I cannot login Nifi UI. Even it doesn't login with anonymous. I get below in logs

<em>==> /data/log/nifi/nifi-user.log <==
2017-10-30 06:11:54,514 WARN [main] o.a.n.a.util.IdentityMappingUtil Identity Mapping property nifi.security.identity.mapping.pattern.kerb was found, but was empty
2017-10-30 06:11:55,605 WARN [main] o.a.n.a.util.IdentityMappingUtil Identity Mapping property nifi.security.identity.mapping.pattern.kerb was found, but was empty
2017-10-30 06:11:55,652 WARN [main] o.a.n.a.util.IdentityMappingUtil Identity Mapping property nifi.security.identity.mapping.pattern.kerb was found, but was empty
2017-10-30 06:13:06,886 INFO [NiFi Web Server-22] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response.
2017-10-30 06:13:07,135 INFO [NiFi Web Server-94] o.a.n.w.a.c.AccessDeniedExceptionMapper anonymous does not have permission to access the requested resource. Unable to view the user interface. Returning Unauthorized response.
2017-10-30 06:14:03,287 INFO [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (eyJhcGxlLG91PU1VTV9NdW1iYWkgSW5kaWEsb3U9QXNpYSxvdT1QZW9wbGUgYW5kIFdvcmtzdGF0aW9ucyxkYz1tb3Jua) GET https://10.248.13.199:9091/nifi-api/flow/current-user (source ip: 10.90.18.237)
2017-10-30 06:14:03,290 INFO [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for cn=Danny Leo,ou=Asia,ou=People and Workstations,dc=ex,dc=com
2017-10-30 06:14:03,293 INFO [NiFi Web Server-20] o.a.n.w.a.c.AccessDeniedExceptionMapper cn=Danny Leo,ou=People,ou=Asia,ou=People and Workstations,dc=ex,dc=com does not have permission to access the requested resource. Unable to view the user interface. Returning Forbidden response.<br></em>

40035-nifi-2.png


It seems I have authorizations/permissions issue. Can you please suggest where I'm missing the configurations.

Thanks,

Suraj


Mentor

@suraj l

I see the below error in the logs

INFO [NiFi Web Server-22] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response. 

Your setup looks kerberized did you configuring a Login Identity Provider that integrates with a Kerberos Key?

<provider>
    <identifier>kerberos-provider</identifier>
    <class>org.apache.nifi.kerberos.KerberosProvider</class>
    <property name="Default Realm">NIFI.APACHE.ORG</property>
    <property name="Kerberos Config File">/etc/krb5.conf</property>
    <property name="Authentication Expiration">12 hours</property>
</provider> 

With the above configuration, username/password authentication can be enabled by referencing this provider in nifi.properties.

nifi.security.user.login.identity.provider=kerberos-provider 

In the Ranger UI your parameter Nifi Resource Identifier, I see you have put * which is open permissions the controlled valid options are

/flow -----Allows users to view the user interface 
/controller 
/tenants 
/site-to-site 
/system 
/proxy 
/counters 

The guide linked correctly states that your NiFi must be configured to run securely (HTTPS) and have an authentication mechanism (user certificates. ldap, or kerberos) in place. Without a secure setup, all users who access the NiFi UI get in with anonymous access which gives all of them full access so all aspects of the NiFi UI.

Remove the http configuration to prevent uncontrolled anonymous access in nifi.properties

@Geoffrey Shelton Okot

I am not using Kerberos. I dont know from where kerberos entries are coming in logs.

Nifi is already configured with SSL. Also I have created policies in Ranger. Please check below screenshot.

40037-ranger-2.png

I have added users in all above policies.

Template for login-identity-providers.xml

 <provider>
            <identifier>ldap-provider</identifier>
            <class>org.apache.nifi.ldap.LdapProvider</class>
            <property name="Identity Strategy">USE_USERNAME</property>
            <property name="Authentication Strategy">SIMPLE</property>
            <property name="Manager DN">CN=hadoop_prd_ad_user,OU=Service Accounts,OU=Hadoop,OU=Servers and Services,DC=ex,DC=com</property>
            <property name="Manager Password">xxx</property>
            <property name="Referral Strategy">FOLLOW</property>
            <property name="Connect Timeout">10 secs</property>
            <property name="Read Timeout">10 secs</property>
            <property name="Url">ldap://ldap.ex.com:389</property>
            <property name="User Search Base">DC=ex,DC=com</property>
            <property name="User Search Filter">sAMAccountName={0}</property>
            <property name="Authentication Expiration">12 hours</property>
            </provider>
nifi.security.user.login.identity.provider=ldap-provider 

In the Ranger UI your parameter Nifi Resource Identifier I have removed * from policy.

authorizers.xml

 <authorizer>
        <identifier>ranger-provider</identifier>


        <class>org.apache.nifi.ranger.authorization.RangerNiFiAuthorizer</class>
        <property name="Ranger Audit Config Path">/usr/hdf/current/nifi/conf/ranger-nifi-audit.xml</property>
        <property name="Ranger Security Config Path">/usr/hdf/current/nifi/conf/ranger-nifi-security.xml</property>
        <property name="Ranger Service Type">nifi</property>
        <property name="Ranger Application Id">nifi</property>
        <property name="Allow Anonymous">false</property>
        <property name="Ranger Admin Identity"></property>


        <property name="Ranger Kerberos Enabled">false</property>

Still my issue is not get resolved. How it is automatically gets logged in with anonymous?

Mentor

@suraj l

Can you upload this log /data/log/nifi/nifi-user.log, meanwhile let me spin up a single node Nifi cluster and to try to reproduce your settings. Do you have some specific document or steps you followed?

Please revert



/data/log/nifi/nifi-user.log

<em>==> /data/log/nifi/nifi-user.log <==
2017-10-30 06:11:54,514 WARN [main] o.a.n.a.util.IdentityMappingUtil Identity Mapping property nifi.security.identity.mapping.pattern.kerb was found, but was empty
2017-10-30 06:11:55,605 WARN [main] o.a.n.a.util.IdentityMappingUtil Identity Mapping property nifi.security.identity.mapping.pattern.kerb was found, but was empty
2017-10-30 06:11:55,652 WARN [main] o.a.n.a.util.IdentityMappingUtil Identity Mapping property nifi.security.identity.mapping.pattern.kerb was found, but was empty
2017-10-30 06:13:06,886 INFO [NiFi Web Server-22] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response.
2017-10-30 06:13:07,135 INFO [NiFi Web Server-94] o.a.n.w.a.c.AccessDeniedExceptionMapper anonymous does not have permission to access the requested resource. Unable to view the user interface. Returning Unauthorized response.
2017-10-30 06:14:03,287 INFO [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (eyJhcGxlLG91PU1VTV9NdW1iYWkgSW5kaWEsb3U9QXNpYSxvdT1QZW9wbGUgYW5kIFdvcmtzdGF0aW9ucyxkYz1tb3Jua) GET https://10.248.13.199:9091/nifi-api/flow/current-user (source ip: 10.90.18.237)
2017-10-30 06:14:03,290 INFO [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for cn=Danny Leo,ou=Asia,ou=People and Workstations,dc=ex,dc=com
2017-10-30 06:14:03,293 INFO [NiFi Web Server-20] o.a.n.w.a.c.AccessDeniedExceptionMapper cn=Danny Leo,ou=People,ou=Asia,ou=People and Workstations,dc=ex,dc=com does not have permission to access the requested resource. Unable to view the user interface. Returning Forbidden response.<br></em>

I have followed multiple links to resolve this issue.

https://community.hortonworks.com/articles/60001/hdf-20-integrating-secured-nifi-with-secured-range....

https://community.hortonworks.com/articles/57980/hdf-20-apache-nifi-integration-with-apache-ambarir....

https://community.hortonworks.com/articles/61729/nifi-identity-conversion.html

Hello @Geoffrey Shelton Okot

Do you find anything related to this issue?

@Geoffrey Shelton Okot

@Matt Clark

Below files are not in Nifi's conf folder.

authorizations.xml

users.xml

Those files are not found anywhere on the system.

Do we need those files ? If yes then how to generate/create both .xml files ?

Also as per https://pierrevillard.com/2017/01/24/integration-of-nifi-with-ldap/comment-page-1/

user's should have specific permissions to access the nifi. I have full permissions to the users but still I'm getting the same permission issue.

What could be the issue?

@Geoffrey Shelton Okot @Pierre Villard @Matt Clarke @Jonas Straub @Yolanda M. Davis

Complete setup scenario: In a cluster ( HDF 3.0.1 - Ambari, Nifi, zookeeper, Ranger, DB - Mysql ), all componants are running fine.

Nifi UI is configured with HTTPS but do not get successful login page in Nifi UI.

(To configure Nifi UI with HTTPS - converted keystore.jks file into pks12 format and loaded the pks12 file into browser)

Ranger is integrated with LDAP successfully.

Ranger UI is accessible through LDAP users.

Copied Nifi's keystore and trustore file from Nifi server to Ranger server to build the trust between them.

(copied at /usr/hdf/current/ranger-admin/conf)

Then Ranger Policy is created and added LDAP users in it.

Also given Read and Write permissions to added LDAP users in Ranger policy.

Note: Ranger UI is not on HTTPS

42428-ranger-policy-to-access-users.png

42429-ranger-1.png

Now there is one issue. If I add some LDAP users in the Ranger policy then I cannot access Nifi UI.

I got 'insuffecient permissions and unable to access the page' kind of errors.

Logs shows authentication is success for LDAP users but authorization is failed. Getting below error as per screenshot.

42431-nifi-2.png

But If I gave {users} in user's section of Ranger Policy, then I can login Nifi UI with my LDAP user.

Also Nifi UI can be accessible by anonymous user. I dont know from where anonymous user is coming.

42430-nifi-1.png

But If I remove {user} from user section then I cannot login with LDAP user as well as anonymous user.

As per some blogs, I found it could be the related from authorizations.xml and users.xml files.

But those files are missing from Nifi servers. How to create/generate those files ?

Nifi Config.

nifi.security.user.login.identity.provider

ldap-provider

Template for login-identity-providers.xml

<provider>
            <identifier>ldap-provider</identifier>
            <class>org.apache.nifi.ldap.LdapProvider</class>
            <property name="Identity Strategy">USE_USERNAME</property>
            <property name="Authentication Strategy">SIMPLE</property>
            <property name="Manager DN">CN=zxc_oi,OU=fox,DC=abc,DC=com</property>
            <property name="Manager Password">xxx</property>
            <property name="Referral Strategy">FOLLOW</property>
            <property name="Connect Timeout">10 secs</property>
            <property name="Read Timeout">10 secs</property>
            <property name="Url">ldap://ldap.abc.com:389</property>
            <property name="User Search Base">DC=abc,DC=com</property>
            <property name="User Search Filter">sAMAccountName={0}</property>
            <property name="Authentication Expiration">12 hours</property>
            </provider>

Please suggest, How I can remove anonymous user by getting logged in Nifi UI.

I have gone through below links and some others as well.

https://community.hortonworks.com/articles/60001/hdf-20-integrating-secured-nifi-with-secured-range....

https://pierrevillard.com/2017/01/24/integration-of-nifi-with-ldap/comment-page-1/#comment-1114

If any more details is required, please let us know.

Mentor

@suraj l

You are right those files ain't in the /usr/hdf/current/nifi/conf no need to generate them because they are encapsulated in the authorizers.xml

<authorizer>
 <identifier>ranger-provider</identifier>
 <class>org.apache.nifi.authorization.FileAuthorizer</class>
 <property name="Authorizations File">/var/lib/nifi/conf/authorizations.xml</property>
 <property name="Users File">/var/lib/nifi/conf/users.xml</property>
 <property name="Initial Admin Identity"></property>
 <property name="Legacy Authorized Users File"></property>

So you should check the values in those files in /var/lib/nifi/conf

@Geoffrey Shelton Okot

Thank you for the clearing the doubt.

But /var/lib/nifi/conf/users.xml and /var/lib/nifi/conf/authorizations.xml files are not created as well.

Can you please suggest what exactly need to do to create them.

Also can you please suggest that, is it possible to remove the anonymous default login ?

Am I searching in right direction, because I have tried multiple ways but still I do not get the expected result. Anything I'm missing in configurations?

Mentor

@suraj l

Policy name error

I noted that you have a policy name proxy instead of /proxy see attached screenshot


NiFi supports user authentication via client certificates, via username/password, or via Apache Knox., Username/password authentication is performed by a Login Identity Provider which is a pluggable mechanism for authenticating users via their username/password it is configured in the nifi.properties file options are LDAP and Kerberos.

The nifi.login.identity.provider.configuration.file=/usr/hdf/current/nifi/conf/login-identity-providers.xml property specifies the configuration file for Login Identity Providers.

The nifi.security.user.login.identity.provider = property indicates which of the configured Login Identity Provider should be used. By default, this property is not configured meaning that username/password must be explicitly enabled.


Anonymous users

Can you edit /etc/nifi/3.0.1.1-5/0/authorizers.xml change the below property from default true to false

<property name="Allow Anonymous">true</property>

Please test and revert


suraj.jpg

@Geoffrey Shelton Okot

/proxy is also tested earlier.

43421-ranger-3.png

Anonymous users is already set to false .

[root@ip- ~]# cat /etc/nifi/3.0.1.1-5/0/authorizers.xml|grep -i anon
        <property name="Allow Anonymous">false</property>
[root@ip-10-248-13-199 ~]#

[root@ip- ~]# cat /usr/hdf/current/nifi/conf/nifi.properties |grep -i login
nifi.login.identity.provider.configuration.file=/usr/hdf/current/nifi/conf/login-identity-providers.xml
nifi.security.user.login.identity.provider=ldap-provider
[root@ip- ~]#



Can you suggest, from where anonymous user is coming? Is it from Ranger policy(LDAP) or is it a default user in Nifi ?

I found the solution. Issue is fixed now.

In my case, one of LDAP username is 'dvteam' but in LDAP database there was full description of username as 'architecture dev team, locations, team details, etc'.

Error messages I found in nifi-user.log. is 'architecture dev team' user was trying to authenticate with nifi nodes. Authentication was successful but authorizations not happening.

The username which I've mentioned in initial admin identity was 'dvteam'.(cn=dvteam,ou=xx,ou=xx,ou=xx,ou=xx,dc=abc,dc=com) Then as per logs, I changed it to (cn=architecture dev team,ou=xx,ou=xx,ou=xx,ou=xx,dc=abc,dc=com)

Also there was some mismatch about host names in node identities section. 'hostname -f' shows a hostname ip-zz-xx-ec2-internal. So, I have given 'ip-zz-xx-ec2-internal' in node identities section but that was not working. Then I have changed the hostnames to 'nifi1.abc.local' and mentioned in node identities.

In 'Template for login-identity-providers.xml' I've made some changes. Earlier I had set 'use_username' in '<property name="Identity Strategy">USE_DN</property>' this section.

later I've changed to use_dn. because as per nifi-user log authentication is happening with LDAP user 'architecture dev team'.

So in my case user_username was not working for authentications.

Every configurations changes I used to remove authorizations.xml and users.xml file from my all nifi nodes.

Also There was confusion on about 'OU' in Node identities section.

What does it mean OU in node identities section? I don't know yet.

Later I've mentioned 'OU=nifi' and also gave host names as 'nifi1.abc.local' , 'nifi2.abc.local', etc.

I have added AD/LDAP user in Initial Admin Identity(cn=architecture dev team,ou=xx,ou=xx,ou=xx,ou=xx,dc=abc,dc=com)

After setting above all, I was facing an error about setting nifi.security.identity.mapping.pattern.dn.

There was a challenge about the pattern definition.

There was 4 'ou' I have defined in initial admin identities and login-identity-providers.xml.

So I've used below pattern and it worked well.

^cn=(.?),ou=(.?),ou=(.?),ou=(.?),ou=(.?),dc=(.?),dc=(.?)$

Note: I have removed Ranger completely.

Thanks,

Suraj

; ;