Support Questions

cjervis · ‎01-09-2020

I have secure nifi and nifi -registry with ldap working fine running on the same machine.

Nifi is running with alb listening on 443 and forwarding to 9999 -- same isntance.

Nifi -registry running with alb listening on 443 and forwarding to 8443 -- same instance.

Dns for nifi -- dev.nifi-test.------------

Dns for nifi -registry - dev.nifi-registry.--------

Imported the nifi-registry cert chain into trustore used by nifi , so no issues with ssl handshake between nifi and nifi-registry.

Intial admin user loaded from LDAP have full access in nifi and was given all access to bucket and can proxy user requests and bucket policies set to full access for this user.

In nifi controller settings are pointed https://dev.nifi-registry...........

But available buckets are not showing up . Please help with this issue.

stevenmatison · ‎01-12-2020

@Chada Can you share more details regarding any specific errors?

This post may have the solution you are looking for:

https://community.cloudera.com/t5/Support-Questions/Getting-Unable-to-obtain-listing-of-buckets-org-...

Chada · ‎01-13-2020

@stevenmatison I tried those earlier. Few things that i checked.making sure user has privileges to the bucket and added hostname in users in nifi -registry with CN=<dns for nifi>,OU=NIFI and made sure that access to bucket and can proxy requests.

Intial user is coming from LDAP provider on both nifi and nifi-registry and hashed userid in both users.xml in nifi and nifi-registry are matching.

Even the public buckets are not loading in nifi.

The nifi-registry url given in the controller settings https://dev.<<<>>.aws.<<<>>.<<>>

If this is given no ssl errors but not able to load any available buckets including public buckets.

I tried adding https://dev.<<<>>.aws.<<<>>.<<>>/nifi-registry in the url here. But throws 404 page not found error and nifi is trying to access /nifi-registry/nifi-registry-api/buckets which is not correct url.

Unable to obtain listing of buckets: org.apache.nifi.registry.client.NiFiRegistryException: Error retrieving all buckets: <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Not Found</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /nifi-registry/nifi-registry-api/buckets. Reason: <pre> Not Found</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.19.v20190610</a><hr/> </body> </html>

Is there a setting or config in nifi that needs to be changed to look for correct path or no path needed just refer to url https://dev.<<<>>.aws.<<<>>.<<>> without any context path?

Nothing from the logs to post. No errors in both nifi and nifi-registry logs

MattWho · ‎01-16-2020

@Chada

Make sure your PrivateKeyEntry being used by your NiFi has both "clientAuth" and "serverAuth" extended key usage.

The NiFi-Registry URL configured in NiFi --> global menu --> controller settings --> registry clients tab will be:

https://<nifi-registry-hostname>:<nifi-registry port>

Your NiFi node(s) will need to exist in NiFi-Registry and be assigned the following special privileges:

Read allow your NiFi nodes to read all buckets to see if new versions of existing version controlled flows exist.
Proxy allows is needed because a NiFi node may proxy requests on behalf of the user authenticated in to NiFi and making NiFi-Registry requests.

Your NiFi authenticated user in NIFi will need to exist in NiFi-Registry (exact string match) and be authorized for a bucket you must first create via the NiFi-Registry's UI as follows:

If your NiFi user has not been authorized to any buckets you will see below error in NiFi when you try version control a process group:

Once the user is properly given authorization to a bucket in NiFi-Registry, you will instead see:

Only buckets for which your user is authorized will show in the bucket pull-down menu.

Hope this helps,

Matt

Yemre · ‎11-12-2021

Hi there,

I have the same issue. Configured LDAP for both Nifi and Registry, both uses admin user which is me.

When I select registry in NiFi, it says no available buckets.

I gave full permission to myself from Registry.

I checked Network tab when starting version control screen in Nifi. Buckets come correctly but seems no permission to write because canWrite property is false.

I tried to make bucket publicly visible but not worked.

Any help would be great pls.

@MattWho

MattWho · ‎11-16-2021

@Yemre

Authorizing your user is not enough.
The NiFi nodes themselves need to be able to successfully authenticate via a mutual TLS handshake with the target NiFi-Registry. Those nodes then need to be authorized to read all buckets and given read/write to proxy user requests.

When a User authenticates in to NiFi, that user entity is authorized to perfrom actions based on authorizations in NiFi. When it comes to NiFi then talking to NiFi-Registry, The NiFi node is proxying request to the NiFi-Registry on behalf of the user authenticated into NiFi.

Also background threads in NiFi just like the NiFi processors added to the canvas are not executing as the user authenticated in to NiFi. So in the background NiFi connects to NiFi-Registry to check on current version controlled process groups to see of newer versions exist.

While you are granting your NiFi nodes the ability to read all buckets, the NiFi users should be given read and write authorizations to the specific buckets that that user is going to sue to version control their Process Group.

@Yemre

The ability to dynamically fetch secrets/passwords form an external source is not something that exists currently. Doing so would require modification with the every component class that uses sensitive properties.

There is some progress in this path however:
https://issues.apache.org/jira/browse/NIFI-5481

This new feature handles pulling secrets from an external vault, but is a NiFi core level feature and does not extend in to individual flow component level.

I recommend raising an Apache NiFi Jira with your specific request.
https://issues.apache.org/jira/projects/NIFI/

If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Matt

Yemre · ‎11-16-2021

Thank you for helping Matt but I have tried to authenticate my Nifi, still not worked. My nifi is not clustered.

I checked the Nifi keystore, console printed this output.

Alias name: 1
Creation date: Mar 6, 2020
Entry type: PrivateKeyEntry
Certificate chain length: 3
Certificate[1]:
Owner: <owner DN>
Issuer: CN=GeoTrust RSA CA 2018, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: <serialnumber>
Valid from: Tue Feb 25 03:00:00 EET 2020 until: Mon Apr 25 15:00:00 EET 2022

I have added the owner DN to Nifi Registry as an user (checked whitespaces carefully). And gave necessary permissions like proxying requests but still no effect.

Yemre · ‎11-19-2021

@MattWho

I have added Nifi keystore Owner (CN=<host>, OU=NIFI) to Registry as an user and gave these privileges. No effect.

It looks like the connection between Nifi and Registry is done correctly because when I made buckets publicly available, the buckets fetched successfuly from Registry. I can see my buckets from Chrome developer tools (Network tab) as above.

So, there is a issue with authorization.

Do I need to save my Nifi Keystore Owner string to LDAP?

Do I need to import Nifi truststore or keystore to Registry stores?

MattWho · ‎11-22-2021

@Yemre

If you have a support contract with Cloudera, I'd recommend opening a support case to assist with your issue here.

Possible causes:
1. Unsuccessful Mutual TLS handshake with NiFi-Registry from the NiFi hosts resulting in NiFi node connection only being 1-way TLS connection and node treated as an "anonymous" user. Anonymous users would could not proxy user requests and can not see anything except public buckets.

--- Caused by missing complete trust chain on one or both sides of connection. Truststore in NiFi-Registry contains complete trust chain for NiFi hosts keystore PrivateKeyEntry.
--- Caused by PrivateKeyEntry not meeting minimum requirements (missing SAN with NiFi hostname, missing EKU of clientAuth, and/or using wildcards are the most common)
2. NiFi-registry is configured with an identity mapping pattern in the nifi-registry.properties file that is matching on the DN from the the NiFi's client certificate presented in the mutual TLS handshake. The Identity mapping value and transform is then being applied which alters the actual client string which must be then authorized for Proxy and buckets policies.

Hope this helps you,
Matt

UOR · ‎05-16-2023

Hello @Yemre,

I'm facing exactly the same issue. How did you manage to solve this ?

Thanks

Cloudera Community

Support Questions

Nifi not able to load available buckets in nifi registry