Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to integrate Nifi with LDAP by using Ranger policy.

How to integrate Nifi with LDAP by using Ranger policy.

Ranger is integrated with LDAP and able to login Ranger UI through domain user.

I've made changes as per
https://pierrevillard.com/2017/01/24/integration-of-nifi-with-ldap/
But still I'm unable to get login

below is the configurations:

Template for login-identity-providers.xml

 <provider>
<identifier>ldap-provider</identifier>
<class>org.apache.nifi.ldap.LdapProvider</class>
<property name="Authentication Strategy">LDAPS</property>
<property name="Manager DN">uid=nifiadmin,ou=nifi,dc=abc,dc=com</property>
<property name="Manager Password">changeitambari</property>
<property name="TLS - Keystore">/usr/hdf/current/nifi/conf/keystore.jks</property>
<property name="TLS - Keystore Password">changeitambari</property>
<property name="TLS - Keystore Type">JKS</property>
<property name="TLS - Truststore">/usr/hdf/current/nifi/conf/truststore.jks</property>
<property name="TLS - Truststore Password">changeitambari</property>
<property name="TLS - Truststore Type">JKS</property>
<property name="TLS - Client Auth">REQUIRED</property>
<property name="TLS - Protocol">TLS</property>
<property name="TLS - Shutdown Gracefully"></property>
<property name="Referral Strategy">FOLLOW</property>
<property name="Connect Timeout">10 secs</property>
<property name="Read Timeout">10 secs</property>
<property name="Url">ldaps://ldap-dallas.abc.com:636</property>
<property name="User Search Base">cn=nifiadmin,ou=nifi,dc=abc,dc=com</property>
<property name="User Search Filter">sAMAccountName={nifiadmin}</property>
<property name="Authentication Expiration">12 hours</property>
</provider>
18 REPLIES 18
Highlighted

Re: How to integrate Nifi with LDAP by using Ranger policy.

Master Guru
@suraj l

Is NiFi displaying the login screen or just blocking access to the UI?

If you are seeing a login screen, what do you see in the nifi-user.log after supplying your username and password?

Thanks,

Matt

Highlighted

Re: How to integrate Nifi with LDAP by using Ranger policy.

Master Guru

@suraj l

Sorry, I am not following what you mean by "It is auto login through anonymous user"

Nifi follows the following Authentication order:

1. Server requests user/client certificate --> if client present valid trusted certificate then it is what will be used to authenticate
*** If above fails the rest of the methods must be configured in NiFi before they will be tried in this order.
2. SPNEGO --> if client presents a user principal via the http connection and it is validated, it will be used for authentication.
3. Either ldap-provider or kerberos-provider --> If both the above fail and a login-identity-provider has been configured, the user will be presented with a login screen which will be used for user authentication.

In your case, you appear to be using LDAPS with in the ldap-provider.
I am curious about your configured "User Search Base" and "User Search Filter" as they appear to be specific to just the nifiadmin user. In that case the the LDAPS search would only ever be able to return credentials for nifiadmin user. If you are trying to login in with some other user, that would explain the "The supplied username and password are not valid".

The next question is what method are you using for user authorization (this occurs after successful authentication.
Options include either NiFi's built-in file based authorizer or an external service like RANGER.

With the default file-based authorizer, you should have configured an "initial admin identity" and "node identities" for each node in your NiFi cluster.

You are saying you actually see the canvas which means some form of authentication and authorization is being successful.

The user log snippet you shared only shows successful authentication for what appears to be the DN for a nifi node? Di you load your nodes cert in to your browser per chance? If not did you authorize "anonymous" in your users.xml and authorizations.xml files?

thanks,

Matt

Highlighted

Re: How to integrate Nifi with LDAP by using Ranger policy.

Is NiFi displaying the login screen or just blocking access to the UI?
>> Nifi displaying login screen. It is not blocking UI
It is auto login through anonymous user.

in logs :

2017-09-27 15:21:32,611 INFO [NiFi Web Server-597] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=ip-10-248-13-29.ec2.internal, OU=NIFI
2017-09-27 15:21:43,402 INFO [NiFi Web Server-547] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for anonymous




Highlighted

Re: How to integrate Nifi with LDAP by using Ranger policy.

Also when I'm trying to log in through nifiadmin or domain user

            The supplied username and password are not valid
Highlighted

Re: How to integrate Nifi with LDAP by using Ranger policy.

Hi Matt,

@Matt
As per your suggestion I have made changes in configurations but still I'm facing the same issue.

Here is the brief information about complete cluster setup:
I'm using HDF 3.0.1.0, Ambari 2.5.1, Nifi 1.2.0, zookeeper 3.4.6, Ranger 0.7.0, ambari metrics 0.1.0.
In this setup I've used total 9 nodes(3-Nifi, 3-zk,1-Ranger,1-db,1-Ambari). Everything is working fine. I've created policy in ranger and through that Nifi UI is accessible. Client certificate is used to access Nifi UI through browser.

Now requirement is,

Nifi and Ranger is need to authenticate with Ldap.

Ranger is authenticated with ldap (still it is not working well with ldaps).

Also Need to integrate Nifi with Ldap.

Currently,

Nifi UI is accessible through client certificate. But I wanted to use ldaps.

So I've disabled client certificate authentication in browser but then Nifi UI is directly login with anonymous user instead of giving login page. I'm struggling to identify the mistake from my configuration. I've attached screenshot of Nifi UI.

Please suggest necessary changes required in this setup.

authorizers.xml

<property name="Node Identity 1">CN=ip-Node1.ec2.internal, OU=NIFI</property>
<property name="Node Identity 2">CN=ip-Node2.ec2.internal, OU=NIFI</property>
<property name="Node Identity 3">CN=ip-Node3.ec2.internal, OU=NIFI</property>


users.xml and Template for login-identity-providers.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
    <groups/>
    <users>
        <user identifier="xxxxencrypted-codexxx" identity="CN=nifiadmin, OU=NIFI"/>
    </users>
</tenants>
<provider>
<identifier>ldap-provider</identifier>
<class>org.apache.nifi.ldap.LdapProvider</class>
<property name="Authentication Strategy">SIMPLE</property>
<property name="Manager DN">uid=nifiadmin,ou=nifi,dc=abc,dc=com</property>
<property name="Manager Password">changeitambari</property>
<property name="TLS - Keystore">/usr/hdf/current/nifi/conf/keystore.jks</property>
<property name="TLS - Keystore Password">changeitambari</property>
<property name="TLS - Keystore Type">JKS</property>
<property name="TLS - Truststore">/usr/hdf/current/nifi/conf/truststore.jks</property>
<property name="TLS - Truststore Password">changeitambari</property>
<property name="TLS - Truststore Type">JKS</property>
<property name="TLS - Client Auth"></property>
<property name="TLS - Protocol">TLS</property>
<property name="TLS - Shutdown Gracefully"></property>
<property name="Referral Strategy">FOLLOW</property>
<property name="Connect Timeout">10 secs</property>
<property name="Read Timeout">10 secs</property>
<property name="Url">ldap://ldap.abc.com:3268</property>
<property name="User Search Base">ou=nifi,dc=abc,dc=com</property>
<property name="User Search Filter">uid={0}</property>
<property name="Authentication Expiration">12 hours</property>
</provider>

Nifi-user.log

2017-09-28 07:52:01,465 WARN [main] o.a.n.a.util.IdentityMappingUtil Identity Mapping property nifi.security.identity.mapping.pattern.dn was found, but was empty
==> nifi-user.log <==
2017-09-28 08:01:11,818 INFO [NiFi Web Server-16] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response.
2017-09-28 08:01:14,101 INFO [NiFi Web Server-21] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (<><CN=ip-10-248-13-29.ec2.internal, OU=NIFI><CN=ip-10-248-12-214.ec2.internal, OU=NIFI>) GET https://ip-10-248-13-29.ec2.internal:8443/nifi-api/flow/current-user (source ip: 10.248.12.214)
2017-09-28 08:01:14,150 INFO [NiFi Web Server-21] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for anonymous


nifi-login-anonymous.png

Thank you,
Suraj

Highlighted

Re: How to integrate Nifi with LDAP by using Ranger policy.

Master Guru

@suraj l

There are a few questions that need to be asked here....

You stated that you installed HDF NiFi and Ranger services. Please provide the current values assigned to the following properties in the nifi.properties file:

nifi.security.user.authorizer=
nifi.security.user.login.identity.provider=

It also may be helpful if you provided your login-identity-providers.xml file and you authorizers.xml file.

Based on your nifi-user.log above a user was never determined during the authentication phase "Attempting request for(<><CN=ip-10-248-13-29.ec2.internal, OU=NIFI><CN=ip-10-248-12-214.ec2.internal, OU=NIFI>)" This breakdown into "<>" <-- user, "<CN=ip-10-248-13-29.ec2.internal, OU=NIFI>" <--nifi node, "<CN=ip-10-248-12-214.ec2.internal, OU=NIFI>)" <-- "NiFi cluster coordinator node. As you can see the user was empty so it is being treated as anonymous.

So Anonymous is what is being passed to the authorizer. Question here is who really is your currently configured authorizer? NiFi's file based authorizer or Ranger?

The file-based authorizer uses the users.xml and authorizations.xml files of which you provided and I do not see anonymous in there anywhere. But if you are using Ranger, these files are not used anyway.

If you are using Ranger, do you have a policy configured that would allow anonymous access?

Thanks,

Matt

Highlighted

Re: How to integrate Nifi with LDAP by using Ranger policy.

@Matt Clarke

1. nifi.properties

nifi.security.user.authorizer=ranger-provider
nifi.security.user.login.identity.provider=ldap-provider

2.login-identity-providers.xml

root@ip-10-248-13-29 conf]# cat /etc/nifi/3.0.1.1-5/0/login-identity-providers.xml





            'Referral Strategy' - Strategy for handling referrals. Possible values are FOLLOW, IGNORE, THROW.
            'Connect Timeout' - Duration of connect timeout. (i.e. 10 secs).
            'Read Timeout' - Duration of read timeout. (i.e. 10 secs).


            'Url' - Url of the LDAP servier (i.e. ldap://<hostname>:<port>).
            'User Search Base' - Base DN for searching for users (i.e. CN=Users,DC=example,DC=com).
            'User Search Filter' - Filter for searching for users against the 'User Search Base'.
            (i.e. sAMAccountName={0}). The user specified name is inserted into '{0}'.


            'Authentication Expiration' - The duration of how long the user authentication is valid
            for. If the user never logs out, they will be required to log back in following
            this duration.
            -->
            <provider>
  <identifier>ldap-provider</identifier>
  <class>org.apache.nifi.ldap.LdapProvider</class>
  <property name="Authentication Strategy">SIMPLE</property>
  <property name="Manager DN">ou=nifi,dc=abc,dc=com</property>
  <property encryption="aes/gcm/256" name="Manager Password">gQvcA6kcKRmaSvOa||4uR1XaOGmnOf9MsmTVfZaBehoyiUF+4/6QimJtTj</property>
  <property name="TLS - Keystore">/usr/hdf/current/nifi/conf/keystore.jks</property>
  <property encryption="aes/gcm/256" name="TLS - Keystore Password">XAWdj01lwKsP7V2m||iKQGPUjmLpLWkgCks5vaoNSI/dlOtbzGXiAuVcJV</property>
  <property name="TLS - Keystore Type">JKS</property>
  <property name="TLS - Truststore">/usr/hdf/current/nifi/conf/truststore.jks</property>
  <property encryption="aes/gcm/256" name="TLS - Truststore Password">bSd27hfNQARojrvf||1AlBXmZ4Fpp8vbNoEO2eywaRR5a1ptv6oThnf6Kd</property>
  <property name="TLS - Truststore Type">JKS</property>
  <property name="TLS - Client Auth"/>
  <property name="TLS - Protocol">TLS</property>
  <property name="TLS - Shutdown Gracefully"/>
  <property name="Referral Strategy">FOLLOW</property>
  <property name="Connect Timeout">10 secs</property>
  <property name="Read Timeout">10 secs</property>
  <property name="Url">ldap://ldap.abc.com:3268</property>
  <property name="User Search Base">dc=abc,dc=com</property>
  <property name="User Search Filter">uid{0}</property>
  <property name="Authentication Expiration">12 hours</property>
</provider>


            <!--
            Identity Provider for users logging in with username/password against a Kerberos KDC server.


            'Default Realm' - Default realm to provide when user enters incomplete user principal (i.e. NIFI.APACHE.ORG).
            'Authentication Expiration' - The duration of how long the user authentication is valid for. If the user never logs out, they will be required to log back in following this duration.
            -->


            <!-- To enable the kerberos-provider remove 2 lines. This is 1 of 2.


            <provider>
            <identifier>kerberos-provider</identifier>
            <class>org.apache.nifi.kerberos.KerberosProvider</class>
            <property name="Default Realm">None</property>
            <property name="Authentication Expiration">12 hours</property>
            </provider>


            To enable the kerberos-provider remove 2 lines. This is 2 of 2. -->




            </loginIdentityProviders>[root@ip-10-248-13-29 conf]#



3. I'm using Ranger. I've given access to below users for that Nifi Policy.

admin, keyadmin, {users}, {owner}, rangeradmin, amb_ranger_admin, rangertagsync

Is there anything, I can do changes in Ranger or edit Policy to get the login page on Nifi UI?

Highlighted

Re: How to integrate Nifi with LDAP by using Ranger policy.

Hello,
Please suggest

Highlighted

Re: How to integrate Nifi with LDAP by using Ranger policy.

@Matt Clarke @Pierre Villard

Latest changes:

authorizers.xml

<authorizer>
        <identifier>ranger-provider</identifier>


        <class>org.apache.nifi.ranger.authorization.RangerNiFiAuthorizer</class>
        <property name="Ranger Audit Config Path">/usr/hdf/current/nifi/conf/ranger-nifi-audit.xml</property>
        <property name="Ranger Security Config Path">/usr/hdf/current/nifi/conf/ranger-nifi-security.xml</property>
        <property name="Ranger Service Type">nifi</property>
        <property name="Ranger Application Id">nifi</property>
        <property name="Allow Anonymous">true</property>
        <property name="Ranger Admin Identity"></property>


        <property name="Ranger Kerberos Enabled">false</property>








<!-- Provide the identity (typically a DN) of each node when clustered (see tool tip for detailed description of Node Identity). Must be specified when Ranger Nifi plugin will not be used for authorization. -->


<property name="Node Identity 1">CN=ip-10-248-13-22.ec2.internal, OU=NIFI</property>
<property name="Node Identity 2">CN=ip-10-248-13-214.ec2.internal, OU=NIFI</property>
<property name="Node Identity 3">CN=ip-10-248-13-29.ec2.internal, OU=NIFI</property>


login-identity-providers.xml

      <provider>
  <identifier>ldap-provider</identifier>
  <class>org.apache.nifi.ldap.LdapProvider</class>
  <property name="Authentication Strategy">LDAPS</property>
  <property name="Manager DN">CN=hadoop_prd,OU=Service Accounts,OU=Hadoop,OU=CORE,OU=Servers and Services,DC=abc,DC=com</property>
  <property encryption="aes/gcm/256" name="Manager Password">xxx</property>
  <property name="TLS - Keystore">/usr/hdf/current/nifi/conf/keystore.jks</property>
  <property encryption="aes/gcm/256" name="TLS - Keystore Password">xx</property>
  <property name="TLS - Keystore Type">JKS</property>
  <property name="TLS - Truststore">/usr/hdf/current/nifi/conf/truststore.jks</property>
  <property encryption="aes/gcm/256" name="TLS - Truststore Password">xx</property>
  <property name="TLS - Truststore Type">JKS</property>
  <property name="TLS - Client Auth">WANT</property>
  <property name="TLS - Protocol">TLS</property>
  <property name="TLS - Shutdown Gracefully"/>
  <property name="Referral Strategy">FOLLOW</property>
  <property name="Connect Timeout">10 secs</property>
  <property name="Read Timeout">10 secs</property>
  <property name="Url">ldap://ldap.xxx.com:3268</property>
  <property name="User Search Base">dc=xxx,dc=com</property>
  <property name="User Search Filter">(cn={0})</property>
  <property name="Authentication Expiration">12 hours</property>
</provider>


Still its not working. users tab is not showing in Nifi UI at burger menu. Screenshot is attached. Please suggest.

39677-nifi-users.png

Don't have an account?
Coming from Hortonworks? Activate your account here