Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar
Master Guru

Apache NiFi supports several authentication mechanisms and provides a pluggable authentication model, allowing anyone to implement a custom identity provider. By default, NiFi always supports two-way SSL as an authentication mechanism, but also provides the ability to use LDAP or Kerberos. This post will demonstrate how to use Kerberos for authentication.

Setting up a KDC

In order to do anything with Kerberos, the first thing we need to do is setup a KDC. For this post we are going to use the *docker-kdc* project which makes it really easy to run a KDC on any environment.

I'm working on OSX and had previously installed docker-machine. The *docker-kdc* project was originally created to work with boot2docker, so to work around that we can do the following:

Create a boot2docker machine:

docker-machine create --driver virtualbox boot2docker 
eval "$(docker-machine env boot2docker)" 

Create /usr/local/bin/boot2docker with the contents provided here https://gist.github.com/sttts/b9c03604be3da01b8d60

We also need to install a library called jq, which I did using homebrew:

brew install jq 

After that we can clone the *docker-kdc* git repo and cd into the directory:

git clone https://github.com/tillt/docker-kdc.git 
cd docker-kdc

At the root of this directory create a file named *kdc.json* which will be the config file used when setting up the KDC:

    { "principals": [
        {
          "id": "bbende@NIFI.APACHE.ORG",
          "password": "bbende"
        },
        {
          "id": "HTTP/nifi.apache.org@NIFI.APACHE.ORG",
          "password": "http"
        }
      ],
      "domain": "nifi.apache.org",
      "realm": "NIFI.APACHE.ORG",
      "ip": "127.0.0.1",
      "port": 48088
    }

This tells *docker-kdc* to create two principals, the first represents the end-user that will authenticate to NiFi, and the second is the principal that NiFi will use to talk to the KDC.

We can now build and start the KDC:

./kdc build 
./kdc start

We can test the KDC by running *./kdc* test and should output similar to the following:

Checking boot2docker active... ok 
Trying to connect to KDC... at 192.168.99.101:48088... ok

As part of starting the KDC, a *krb5.conf* and *krb5.keytab* were generated in the *docker-kdc* directory.

The krb5.conf is the configuration file that clients would use to talk to the KDC running in the docker container, and krb5.keytab contains the keytabs for the principals that were specified in kdc.json.

In order to setup the current shell to leverage the krb5.conf generated by *docker-kdc*, and not the one in /etc, run the following command:

$(./kdc shellinit)

We should now be able to perform a kinit against the KDC using one of the principals that we specified:

kinit bbende@NIFI.APACHE.ORG 

Performing a klist should show something like the following:

klist 
Credentials cache: API:5D669ADD-9028-441F-A6E0-C2C78DA04CC5
            Principal: bbende@NIFI.APACHE.ORG
      Issued                Expires               Principal
    Aug 31 10:12:29 2016  Aug 31 20:12:27 2016  krbtgt/NIFI.APACHE.ORG@NIFI.APACHE.ORG

One last bit of setup is that we will have to access NiFi using nifi.apache.org as the hostname in order to line up with the domain in our KDC.

Edit */etc/hosts* and map nifi.apache.org to localhost:

127.0.0.1localhost nifi.apache.org

Configuring NiFi

In order to perform any type of authentication, NiFi has to be configured to use https. One way to generate a certificate and truststore is using the NiFi Toolkit, you can read how to use the toolkit in the Apache NiFi Admin Guide.

The nifi.properties related to https would look something like the following:

    nifi.remote.input.secure=true
    nifi.web.https.host=localhost
    nifi.web.https.port=9445
    nifi.security.keystore=./conf/keystore.jks
    nifi.security.keystoreType=jks
    nifi.security.keystorePasswd=<your password>
    nifi.security.keyPasswd=<your password>
    nifi.security.truststore=./conf/truststore.jks
    nifi.security.truststoreType=jks
    nifi.security.truststorePasswd=<your password>

We also need to set some additional properties related to Kerberos:

    nifi.security.user.login.identity.provider=kerberos-provider
    nifi.kerberos.krb5.file=/path/to/docker-kdc/krb5.conf
    nifi.kerberos.spnego.principal=HTTP/nifi.apache.org@NIFI.APACHE.ORG
    nifi.kerberos.spnego.keytab.location=/path/to/docker-kdc/krb5.keytab
    nifi.kerberos.spnego.authentication.expiration=12 hours

The login identity provider is the identifier of a provider defined in login-identity-providers.xml. We need to edit *login-identity-providers.xml* and uncomment the kerberos-provider:

    <provider>
        <identifier>kerberos-provider</identifier>
        <class>org.apache.nifi.kerberos.KerberosProvider</class>
        <property name="Default Realm">NIFI.APACHE.ORG</property>
        <property name="Authentication Expiration">12 hours</property>
    </provider>

The last thing we need to do is configure the Intial Admin Identity in *authorizers.xml*:

    <property name="Initial Admin Identity">bbende@NIFI.APACHE.ORG</property>

If we were using certificates to authenticate to NiFi, we would put the DN from a certificate here, but since we are going to authenticate with Kerberos, the admin needs to be the principal for our end user.

At this point we can start NiFi:

    ./bin/nifi.sh start

Kerberos Login

There are two ways we can authenticate to NiFi using Kerberos:

Kerberos Login - NiFi presents a login screen and authenticates the user against the KDC.

SPNEGO - A ticket that was established outside of NiFi is forwarded to NiFi through the browser.

If we navigate to NiFi in our browser at https://nifi.apache.org:9445/nifi, assuming we didn't already configure our browser for SPNEGO, then we should get prompted with a login screen:

7153-01-nifi-login.png

NOTE: If you have a client certificate in your browser that corresponds to this NiFi instance, you may get prompted to use that certificate, but we want to decline in this case so that we end up at the login page.

At this login screen we can enter the principal name and password (in this example bbende and bbende) and we should get authenticated and see the NiFi UI:

7154-02-nifi-sso.png

SPNEGO

Using SPNEGO requires some extra configuration in your browser. I am using Firefox which requires going to the about:config page and setting the following properties:

7155-03-firefox-config.png

NOTE: The trusted URIs are the host and port being used to access NiFi in the browser, which in this case is https://nifi.apache.org:9445. This works because of the mapping we setup earlier to map nifi.apache.org to localhost.

If you followed the steps earlier, you probably already have a ticket, but you can check by issuing klist. If you don't see a ticket for your user principal then you need to perform a kinit:

kinit bbende@NIFI.APACHE.ORG 

You should then be able to go to the NiFi UI in your browser without requiring a login.

9,816 Views
Comments
avatar
Expert Contributor

Hi @Bryan Bende,

Thank you for this tutorial. Very helpful.

May I ask do we need to also set up below properties in a cluster mode?

nifi.kerberos.service.principal

nifi.kerberos.service.keytab.location

Is kerberos.service and kerberos.spnego one or another? Or both must be set? Since I saw some examples set up kerberos.service rather than kerberos.spnego, e.g. https://community.hortonworks.com/articles/34147/nifi-security-user-authentication-with-kerberos.htm...

Thanks.

avatar
Master Guru

If you are setting up authentication for users accessing NiFi's UI, then you only need the spnego properties as shown in this post.

If you need NiFi to authenticate to other services, for example to talk Ranger when Ranger is kerberized, then you need the service principal and keytab.

avatar
Expert Contributor

Hi @Bryan Bende,

It seems the authentication works now.

But I have issue with authorization. I used the file-based authorization approach.

After I type username and password, it says "Unknown user with identity 'alvin@NIFI.COM'. Contact the system administrator." instead of prompting out "Justification" page.

Any idea? Thanks.

2017-09-08 19:48:33,294 INFO [NiFi Web Server-29] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for alvin@NIFI.COM 2017-09-08 19:48:33,295

INFO [NiFi Web Server-29] o.a.n.w.a.c.AccessDeniedExceptionMapper identity[alvin@NIFI.COM], groups[none] does not have permission to access the requested resource. Unknown user with identity 'alvin@NIFI.COM'. Returning Forbidden response.

2017-09-08 19:48:33,295 DEBUG [NiFi Web Server-29] o.a.n.w.a.c.AccessDeniedExceptionMapper org.apache.nifi.authorization.AccessDeniedException: Unknown user with identity 'alvin@NIFI.COM'. at org.apache.nifi.web.api.FlowResource.authorizeFlow(FlowResource.java:230) at org.apache.nifi.web.api.FlowResource.getCurrentUser(FlowResource.java:316)

avatar
Master Guru

Something is not set up correctly because the log is showing alvin@NIFI.COM in some places and alvin@NIFI.COM in other places.

What are you entering as the username when you login?

What is entered for the Initial Admin in authorizers.xml?

avatar
Expert Contributor

Hi @Bryan Bende,

I deleted my previous replies since it's misleading.

Let me restate the current issue clearly.

The reason I saw below error

"Unknown user with identity 'alvin@NIFI.COM'. Contact the system administrator."

is due to I login Initial Admin account to add user "CN=alvin, OU=NIFI.COM" in NiFi web.

The correct way is to add 'alvin@NIFI.COM' and set its policy.

After that my issue is gone.

Thanks.

avatar
Expert Contributor

Hi @Bryan Bende

I have a new issue when I tried to login Initial Admin Identity by using username and password.

In previous test, I successfully made the client cert login work.

I have an principle "admin@NIFI.COM" in kerberos, and "admin@NIFI.COM" as Initial Admin in authorizers.xml

When I login as "admin", the user log shows:

2017-09-28 14:27:06,112 INFO [NiFi Web Server-120] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for admin@NIFI.COM

2017-09-28 14:32:13,193 ERROR [NiFi Web Server-19] o.a.nifi.web.security.jwt.JwtService There was an error validating the JWT io.jsonwebtoken.JwtException: Unable to validate the access token.

Caused by: io.jsonwebtoken.SignatureException: JWT signature does not match locally computed signature. JWT validity cannot be asserted and should not be trusted.

When the nifi cluster is up, I can login in as "admin". After logout, and login again, NiFi gives me above error.

Is it caused by the browser cache the token?

Thanks.

avatar
Expert Contributor

Hi @Bryan Bende this tutorial is really helpful, thank you!
For me, everything is working, except the "SPNEGO" part:
a) When I open "https://myhost.de:9445/nifi" (without a kinit before), my Kerberos Client asks for authentication (which looks good). When I enter the principal and password, it continues with step b.

b) When I already made a "kinit" before opening my browser and entering "https://myhost.de:9445/nifi", I always get the username / password prompt as shown in section "Kerberos Login".

What am I missing here? I configured the following settings in my Firefox browser:

Tested it on Centos7, Ubuntu and Windows, I always get the login screen, instead of skipping it after the "kinit". Can you help?

avatar
Master Guru

No problem, glad it was helpful 🙂

If you are reaching the login screen, then it means your browser is not forwarding your credentials to NiFi.

You could try setting the negotiate properties to just "myhost.de" instead of the full url with port.

Another thing to look at might be the domain being used by your KDC...

In this example I was using nifi.apache.org as the domain, so I had to add a mapping in /etc/hosts to map nifi.apache.org to localhost so I could use nifi.apache.org in my browser to access my local NiFi.

If you are accessing myhost.de to get to your NiFi instance, but that isn't the domain in your KDC, then it won't line up and probably won't forward your credentials.

Version history
Last update:
‎08-17-2019 10:26 AM
Updated by:
Contributors