Created 05-28-2018 09:21 AM
Hello,
I would like to use NiFi and NiFi Registry in secured context (with LDAP auth) on my server. User login with credentials works, but I receive the following error when I attempt to get buckets from NiFi Registry:
"Unable to obtain listing of buckets: org.apache.nifi.registry.client.NiFiRegistryException: Error retrieving all buckets: An Authentication object was not found in the SecurityContext Contact the system administrator."My set-up:
version: '3' services: nifi: image: apache/nifi:latest ports: - "8443:8443" volumes: - /srv/nifi/certs:/opt/certs environment: AUTH: ldap KEYSTORE_PATH: /opt/certs/keystore.p12 KEYSTORE_TYPE: PKCS12 KEYSTORE_PASSWORD: changed TRUSTSTORE_PATH: /opt/certs/truststore.jks TRUSTSTORE_PASSWORD: changed TRUSTSTORE_TYPE: JKS INITIAL_ADMIN_IDENTITY: 'foo.bar' LDAP_AUTHENTICATION_STRATEGY: 'SIMPLE' LDAP_MANAGER_DN: 'cn=admin,dc=mydomain,dc=com' LDAP_MANAGER_PASSWORD: 'changed' LDAP_USER_SEARCH_BASE: 'ou=people,dc=mydomain,dc=com' LDAP_USER_SEARCH_FILTER: 'uid={0}' LDAP_IDENTITY_STRATEGY: 'USE_USERNAME' LDAP_URL: 'ldap://my-ip:389' networks: - nifi nifi-registry: image: apache/nifi-registry:latest ports: - "18443:18443" volumes: - /srv/nifi/certs:/opt/certs environment: AUTH: ldap KEYSTORE_PATH: /opt/certs/keystore.p12 KEYSTORE_TYPE: PKCS12 KEYSTORE_PASSWORD: changed TRUSTSTORE_PATH: /opt/certs/truststore.jks TRUSTSTORE_PASSWORD: changed TRUSTSTORE_TYPE: JKS INITIAL_ADMIN_IDENTITY: 'foo.bar' LDAP_AUTHENTICATION_STRATEGY: 'SIMPLE' LDAP_MANAGER_DN: 'cn=admin,dc=mydomain,dc=com' LDAP_MANAGER_PASSWORD: 'changed' LDAP_USER_SEARCH_BASE: 'ou=people,dc=mydomain,dc=com' LDAP_USER_SEARCH_FILTER: 'uid={0}' LDAP_IDENTITY_STRATEGY: 'USE_USERNAME' LDAP_URL: 'ldap://my-ip:389' networks: - nifi networks: nifi: driver: bridge
* LDAP server using SIMPLE authentication
* nginx on one server with two subdomains:NiFi subdomain: server { listen 443 ssl; server_name nifi.mydomain.com; access_log /var/log/nginx/nifi.mydomain.com_access.log; error_log /var/log/nginx/nifi.mydomain.com_error.log; ssl_certificate /srv/certs/fullchain.pem; ssl_certificate_key /srv/certs/privkey.pem; location ^~ /.well-known/acme-challenge/ { allow all; root /var/www; } location / { proxy_pass https://localhost:8443/; } } NiFi Registry: server { listen 443 ssl; server_name registry.mydomain.com; access_log /var/log/nginx/registry.mydomain.com_access.log; error_log /var/log/nginx/registry.mydomain.com_error.log; ssl_certificate /srv/certs/fullchain.pem; ssl_certificate_key /srv/certs/privkey.pem; location ^~ /.well-known/acme-challenge/ { allow all; root /var/www; } location / { proxy_pass https://localhost:18443/; } }
Is there any config that I have missed? Thanks!
Created 05-28-2018 04:49 PM
If I remember correctly, you will need to add Nifi Registry Server DN (SSL machine name) to Nifi > Access Policies and Nifi Server DN to Nifi Registry to be able to read and query buckets. Once they both know each other, buckets will load.
Created 05-29-2018 01:44 PM
@Umair Khan Thank you for the answer!
Can you give a concrete example, please?
Need I create a new user (or group?) with DN as login, like "CN=*.mydomain.org" (I have a wildcard certificate) in NiFi/NiFi Registry and set policies ("proxy user requests"), right?
Created 05-29-2018 02:05 PM
You will need machine names (DN) for both Nifi and Nifi Registry like "CN=machinename,...dc=example,dc=com" where in CN you will use server name the remaining portion will come under whatever wildcard (sub)domain you have.
Then you will enter the full DN, manually as a user in both Nifi and Nifi Registry. This method is similar to when setting up site-to-site policies here:
Finally, you can check logs files for errors when pulling buckets in both Nifi and Nifi registry in files *-app.log and *-user.log, this may as well give you a full DN Nifi is looking for.
Created 06-14-2018 09:35 AM
Thank you very much for your help, but it doesn't work for me 😕
I have checked DN of my keystore.p12 with this command "keytool -list -keystore keystore.p12 -storepass {mypass} -storetype PKCS12 -v":
... Creation date: Jun 14, 2018 Entry type: PrivateKeyEntry Certificate chain length: 2 Certificate[1]: Owner: CN=*.mydomain.org Issuer: CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US ...
and added new NiFi user "CN=*.mydomain.org".
If i check nifi-user.log after mentioned error, it looks like this (it's looking for my LDAP account "max.musti"):
"... 2018-06-14 09:03:39,320 INFO [NiFi Web Server-22802] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for max.musti 2018-06-14 09:03:39,435 INFO [NiFi Web Server-22823] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for () GET https://nifi.mydomain.org/nifi-api/flow/registries/ac177c35-0163-1000-5bd9-f957d9a80534/buckets (source ip: 192.168.32.1) 2018-06-14 09:03:39,436 INFO [NiFi Web Server-22823] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for max.musti 2018-06-14 09:03:39,522 INFO [NiFi Web Server-22823] o.a.n.w.a.config.NiFiCoreExceptionMapper org.apache.nifi.web.NiFiCoreException: Unable to obtain listing of buckets: org.apache.nifi.registry.client.NiFiRegistryException: Error retrieving all buckets: An Authentication object was not found in the SecurityContext Contact the system administrator. . Returning Conflict response."
Maybe there is something wrong with nginx settings (proxy?), docker configs or LDAP, i have tried a lot of different configurations, but currently without success.
I would be very thankful for further suggestions.
Created 11-19-2018 03:13 PM
Couple things about your certificates:
The certificates must include 1 or more SubjectAlternativeNames (SAN) for security reasons. Since you are using a wildcard in the DN for the certificate owner, You should have a unique SAN entry for each server hostname that is using this certificate.
You also must make sure that the certificates support being used for both "clientAuth' and 'serverAuth'. You often can see this called out in the verbose key output from keytool in the ExtendedKeyUsage section.
*** NiFi can act as both a client (such as when using Remote Process Groups or talking to NiFi-registry) and a server.
-
Alternatively and recommended, you could create a separate certificate for each of your servers (these will still require a SAN entry).
-
When it comes to NiFi talking to registry, the following must be successful:
1. A successful 2-way TLS handshake between NiFi and NIFi-registry. I think this may be were you are having an issue. Specifically with your NiFi server(s) presenting a client cert to the NiFi-registry. (This is where the "clientAuth" extendedKeyUsage comes in to the picture)
2.The client server(s) must all be authorized for both "Read" on "Can Manage buckets" and "Can proxy user requests".
-
Hope this helps you get your issues resolved.
-
Thank you,
Matt
-