Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NiFi and NiFi Registry authentication error

avatar
New Contributor

Hello,

I would like to use NiFi and NiFi Registry in secured context (with LDAP auth) on my server. User login with credentials works, but I receive the following error when I attempt to get buckets from NiFi Registry:

"Unable to obtain listing of buckets: org.apache.nifi.registry.client.NiFiRegistryException: Error retrieving all buckets: An Authentication object was not found in the SecurityContext Contact the system administrator."
My set-up:
* docker-compose with nifi and nifi-registry:
version: '3'

services:

  nifi:
    image: apache/nifi:latest
    ports:
      - "8443:8443"
    volumes:
      - /srv/nifi/certs:/opt/certs
    environment:
      AUTH: ldap
      KEYSTORE_PATH: /opt/certs/keystore.p12
      KEYSTORE_TYPE: PKCS12
      KEYSTORE_PASSWORD: changed
      TRUSTSTORE_PATH: /opt/certs/truststore.jks
      TRUSTSTORE_PASSWORD: changed
      TRUSTSTORE_TYPE: JKS
      INITIAL_ADMIN_IDENTITY: 'foo.bar'
      LDAP_AUTHENTICATION_STRATEGY: 'SIMPLE'
      LDAP_MANAGER_DN: 'cn=admin,dc=mydomain,dc=com'
      LDAP_MANAGER_PASSWORD: 'changed'
      LDAP_USER_SEARCH_BASE: 'ou=people,dc=mydomain,dc=com'
      LDAP_USER_SEARCH_FILTER: 'uid={0}'
      LDAP_IDENTITY_STRATEGY: 'USE_USERNAME'
      LDAP_URL: 'ldap://my-ip:389'
    networks:
      - nifi

  nifi-registry:
    image: apache/nifi-registry:latest
    ports:
      - "18443:18443"
    volumes:
      - /srv/nifi/certs:/opt/certs
    environment:
      AUTH: ldap
      KEYSTORE_PATH: /opt/certs/keystore.p12
      KEYSTORE_TYPE: PKCS12
      KEYSTORE_PASSWORD: changed
      TRUSTSTORE_PATH: /opt/certs/truststore.jks
      TRUSTSTORE_PASSWORD: changed
      TRUSTSTORE_TYPE: JKS
      INITIAL_ADMIN_IDENTITY: 'foo.bar'
      LDAP_AUTHENTICATION_STRATEGY: 'SIMPLE'
      LDAP_MANAGER_DN: 'cn=admin,dc=mydomain,dc=com'
      LDAP_MANAGER_PASSWORD: 'changed'
      LDAP_USER_SEARCH_BASE: 'ou=people,dc=mydomain,dc=com'
      LDAP_USER_SEARCH_FILTER: 'uid={0}'
      LDAP_IDENTITY_STRATEGY: 'USE_USERNAME'
      LDAP_URL: 'ldap://my-ip:389'
    networks:
      - nifi


networks:
  nifi:
    driver: bridge

* LDAP server using SIMPLE authentication

* nginx on one server with two subdomains:
NiFi subdomain:

server
{
        listen          443 ssl;
        server_name nifi.mydomain.com;

        access_log      /var/log/nginx/nifi.mydomain.com_access.log;
        error_log       /var/log/nginx/nifi.mydomain.com_error.log;

        ssl_certificate         /srv/certs/fullchain.pem;
        ssl_certificate_key     /srv/certs/privkey.pem;

        location ^~ /.well-known/acme-challenge/ { allow all; root /var/www; }
        location /
        {
        	proxy_pass	https://localhost:8443/;
        }
}

NiFi Registry:
server
{
        listen          443 ssl;
        server_name     registry.mydomain.com;

        access_log      /var/log/nginx/registry.mydomain.com_access.log;
        error_log       /var/log/nginx/registry.mydomain.com_error.log;

        ssl_certificate         /srv/certs/fullchain.pem;
        ssl_certificate_key     /srv/certs/privkey.pem;

        location ^~ /.well-known/acme-challenge/ { allow all; root /var/www; }
        location /
        {
		    proxy_pass	https://localhost:18443/;
        }
}


Is there any config that I have missed? Thanks!

5 REPLIES 5

avatar
Expert Contributor

If I remember correctly, you will need to add Nifi Registry Server DN (SSL machine name) to Nifi > Access Policies and Nifi Server DN to Nifi Registry to be able to read and query buckets. Once they both know each other, buckets will load.

avatar
New Contributor

@Umair Khan Thank you for the answer!

Can you give a concrete example, please?
Need I create a new user (or group?) with DN as login, like "CN=*.mydomain.org" (I have a wildcard certificate) in NiFi/NiFi Registry and set policies ("proxy user requests"), right?

avatar
Expert Contributor

You will need machine names (DN) for both Nifi and Nifi Registry like "CN=machinename,...dc=example,dc=com" where in CN you will use server name the remaining portion will come under whatever wildcard (sub)domain you have.

Then you will enter the full DN, manually as a user in both Nifi and Nifi Registry. This method is similar to when setting up site-to-site policies here:

https://community.hortonworks.com/articles/88473/site-to-site-communication-between-secured-https-a....

Finally, you can check logs files for errors when pulling buckets in both Nifi and Nifi registry in files *-app.log and *-user.log, this may as well give you a full DN Nifi is looking for.

avatar
New Contributor

Thank you very much for your help, but it doesn't work for me 😕

I have checked DN of my keystore.p12 with this command "keytool -list -keystore keystore.p12 -storepass {mypass} -storetype PKCS12 -v":

...
Creation date: Jun 14, 2018
Entry type: PrivateKeyEntry
Certificate chain length: 2
Certificate[1]:
Owner: CN=*.mydomain.org
Issuer: CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US
...

and added new NiFi user "CN=*.mydomain.org".


If i check nifi-user.log after mentioned error, it looks like this (it's looking for my LDAP account "max.musti"):

"...
2018-06-14 09:03:39,320 INFO [NiFi Web Server-22802] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for max.musti
2018-06-14 09:03:39,435 INFO [NiFi Web Server-22823] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for () GET https://nifi.mydomain.org/nifi-api/flow/registries/ac177c35-0163-1000-5bd9-f957d9a80534/buckets (source ip: 192.168.32.1)
2018-06-14 09:03:39,436 INFO [NiFi Web Server-22823] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for max.musti
2018-06-14 09:03:39,522 INFO [NiFi Web Server-22823] o.a.n.w.a.config.NiFiCoreExceptionMapper org.apache.nifi.web.NiFiCoreException: Unable to obtain listing of buckets: org.apache.nifi.registry.client.NiFiRegistryException: Error retrieving all buckets: An Authentication object was not found in the SecurityContext Contact the system administrator.
. Returning Conflict response."


Maybe there is something wrong with nginx settings (proxy?), docker configs or LDAP, i have tried a lot of different configurations, but currently without success.

I would be very thankful for further suggestions.

avatar
Super Mentor

@Max Musti

Couple things about your certificates:

The certificates must include 1 or more SubjectAlternativeNames (SAN) for security reasons. Since you are using a wildcard in the DN for the certificate owner, You should have a unique SAN entry for each server hostname that is using this certificate.

You also must make sure that the certificates support being used for both "clientAuth' and 'serverAuth'. You often can see this called out in the verbose key output from keytool in the ExtendedKeyUsage section.

*** NiFi can act as both a client (such as when using Remote Process Groups or talking to NiFi-registry) and a server.

-

Alternatively and recommended, you could create a separate certificate for each of your servers (these will still require a SAN entry).

-

When it comes to NiFi talking to registry, the following must be successful:

1. A successful 2-way TLS handshake between NiFi and NIFi-registry. I think this may be were you are having an issue. Specifically with your NiFi server(s) presenting a client cert to the NiFi-registry. (This is where the "clientAuth" extendedKeyUsage comes in to the picture)

2.The client server(s) must all be authorized for both "Read" on "Can Manage buckets" and "Can proxy user requests".

-

Hope this helps you get your issues resolved.

-

Thank you,

Matt

-