Skip to main content
SandyClouds
SandyClouds
Gold ⭐️
July 5, 2023
Solved

No available buckets after secure Nifi connected to secure Nifi registry

Hello All,

 

I had secure Nifi and Secured Nifi-registry running on Docker.

Both share same certificates

Able to access both UI's individually.

 

Given static IP to Registry in Docker and subjectalternativenames with this IP while generating certificates.

 

Given this static_IP:<port> in registry client.

created localhost user : CN=localhost, OU=Nifi in both systems and given necessary permissions.

Created few buckets and given all necessary permissions..

 

SandyClouds_3-1688552216202.png

 

 

SandyClouds_0-1688552056963.png

 

SandyClouds_1-1688552101725.png

 

 

 

SandyClouds_2-1688552139230.png

 

 

But when I try to do version control, it says no available buckets.

 

SandyClouds_4-1688552260460.png

 

 

As i troubleshoot, I can see the logs that shows successful authorization to registry..

 

2023-07-05 10:25:47,312 INFO [NiFi Web Server-468] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 10.5.0.1 [CN=new_admin, OU=Nifi] GET https://localhost:8443/nifi-api/flow/registries/21d17428-0189-1000-4a0e-a91cc8cd2aa7/buckets
2023-07-05 10:25:47,312 INFO [NiFi Web Server-468] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [CN=new_admin, OU=Nifi] 10.5.0.1 GET https://localhost:8443/nifi-api/flow/registries/21d17428-0189-1000-4a0e-a91cc8cd2aa7/buckets
2023-07-05 10:26:11,541 INFO [NiFi Web Server-457] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 10.5.0.1 [CN=new_admin, OU=Nifi] GET https://localhost:8443/nifi-api/flow/process-groups/219e3cc2-0189-1000-7c6f-90a4496cf140
2023-07-05 10:26:11,541 INFO [NiFi Web Server-457] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [CN=new_admin, OU=Nifi] 10.5.0.1 GET https://localhost:8443/nifi-api/flow/process-groups/219e3cc2-0189-1000-7c6f-90a4496cf140

 

 

when i tried to open the buckets API URL (from above log) in my browser, i could get the buckets list aslo..

 

highlighted my buckets in Nifi-registry

SandyClouds_5-1688552930787.png

 

I wonder why UI is not able to show the list, when Nifi's internal API call is able to fetch them correctly.

 

Does this indicate a bug from Nifi ? or I am missing something ?

 

After going through all the hurdles and still not successful with Integration, I am thingking about skipping secure installtions and just proceed with unsecure Implementations...

Because this is running in VPN on a server, I need to ssh tunnel it anyway to even access the UI. So I was thinking it is okay for unsecure Nifi UIs (as this is internal).

 

Please share your thoughts as well for my understanding. 

Please let know if any more information is needed in this question for better understanding.

 

Thank You..

 

 

    MattWho
    MattWhoAnswer
    Titanium ⭐️⭐️⭐️
    July 7, 2023

    @SandyClouds 
    What i am saying is that rest-api call response you got via command line is probably not the exact same request made by NiFi.   What that response json tells you is that the user that made the request only has READ to all three buckets. You did not share the complete rest-api call you made, so I am guessing you did not pass any user authentication token or clientAuth certificate in yoru rest-api request and this that request was treated as bing made by an anonymous user.  Since your bucket(s) have the "make publicly visible" checked, anonymous user will have only READ access to the buckets.

    In NiFi you are trying to start version control on a Process group.  That means that the NiFi node certificate needs to be authorized on all buckets and authorized to proxy the NiFi authenticated user (who also needs to have read, write on the specific bucket to start version control.  If the response received back to NiFi during the request is that nodes or user on has READ, then no buckets would be shown in that UI since the action that UI is performing would be a WRITE.  

    You should use the "developer tools" in your web browser to capture the actual full request  being made by NiFi when you try to start version control on a process group.  Most developer tolls even give you and option to "copy as curl" which you can then just paste in a cmd window and execute manually yourself outside of NiFi to see the same response NiFi is getting.

    My guess here is that you are having a TLS handshake issue resulting in yoru requests to NiFi-Registry being anonymous requests.   The NiFi logs are not going to show you details on what authorization was on the NiFi-Registry side (you'll need to check the nifi-registry-app.log for that).  All the NiFi log you shared shows is that your "new_admin" was authorized to execute the NiFi rest-api endpoint.  This triggering the connection attempt with NiFi-Registry which also appeared successful (likely as anonymous).

    If you look at the verbose output for the NiFi keystore, NiFi truststore, NiFi-Registry keystore, and NiFi-Registry truststore, you'll be able to determine if a mutual TLS exchange would even be possible between the two services using the keystore and truststores you have in place.  These versbose outputs which you can get using the keytool command will also show you the EKUs and SANs present on your certificates.

    <path to JDK>/keytool -v -list -keystore <keystore or truststore file>

    When prompted for password, you can just hit enter without providing password.

    The screenshots you shared showing the authorizations configured in NiFi-Registry are sufficient.  I was simply pointing out that you gave too much authorization to your NiFi node in the global policy section.   This is why i am leaning more to a TLS handshake issue resulting in an anonymous user authentication rather then an authorization issue.  You'll also want to inspect your nifi-registry-app.log for the authorization request coming from NiFi to see if is authorizing "anonymous" or some other string.  Perhaps you have some identity mapping pattern configured on NiFi-Registry side that is trimming out the CN  from your certificate DNs?  In that case you could also see this behavior because your NiFi-Registry would be looking up authorization on different identity string (instead of "CN=new_admin, OU=Nifi, i maybe looking for just "new_admin").  NiFi and NiFi-Registry authorizations are exact string matches (case sensitive); otherwise, treated as different users.

    If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

    Thank you,

    Matt

    View in thread

    3 replies

    MattWho
    Titanium ⭐️⭐️⭐️
    MattWho
    Titanium ⭐️⭐️⭐️
    July 5, 2023

    @SandyClouds 

    The rest-api output does nit match with your other image's authorizations.  I am going to guess that the rest-api call you made did not include a certificate?  Your bucket image shared shows that the bucket "my_bucket" was created as a "public" bucket meaning that anyone can view that bucket and the flows stored within that bucket (Rest-Api call returned "canRead: true" and canWrite: false", i'd expect both of these to return true if you passed a certificate for localhost or new_admin as your other image shows read, write, and deleted authorized for those users on "my_bucket". 

    In order to see a bucket in NiFi when starting version control on a process group, The authenticated user in NiFi must be authorized "write" on the bucket.   

    The permission required are:
    1. NiFi node certificate must have "clientAuth and serverAuth extendedKeyUsage" and that clientAuth certificate needs to be authorized for Read,Write, Delete on "Can Manage Bucket" and "Can Proxy User Requests".  NO other authorizations are needed for the NiFi node certificate.
    2.  The Authenticated User in NiFi which is being proxied via the NiFi node certificate needs to be authorized for the following on any bucket that users wants to use:

       2a:  READ - ability to import a version controlled flow from the NiFi-Registry bucket.
       2b:  WRITE - ability to start version control or commit new version of a flow to NiFi-Registry
       2c:  DELETE - ability to delete a flow from a bucket via the NiFi-Registry UI.

    So I would start by getting a verbose listing of the keystore used in nifi.properties file to make sure that the PrivateKeyEntry has clientAuth as an ExtendedKeyUsage (EKU).

    Then I would uncheck the "make publicly visible" box for your "my_bucket" and make your rest-api call again to see what permissions your NiFi node certificate (localhost) and new_admin have.  Remember that what is returned by that call dictates what will show in the various UIs within NiFi.  

    If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

    Thank you,

    Matt

     

    Translate
    SandyClouds
    SandyCloudsAuthor
    Gold ⭐️
    July 7, 2023

    Thanks @MattWho for taking time to answer this.

    I created certificates using tls toolkit like below:

    ./bin/tls-toolkit.sh standalone -n localhost -C 'CN=admin,OU=NiFi' --subjectAlternativeNames <IP of Docker Registry> -o ./certs

     

    "1. NiFi node certificate must have "clientAuth and serverAuth extendedKeyUsage" and that clientAuth certificate needs to be authorized for Read,Write, Delete on "Can Manage Bucket" and "Can Proxy User Requests".  NO other authorizations are needed for the NiFi node certificate."

    Dos it mean, i need to recreate in different way ? or something wrong in above command..?

     

    My rest api call is something i saw in the logs and just typed in browser and it worked and gave me above json as result.

    Translate
    MattWho
    Titanium ⭐️⭐️⭐️
    MattWhoAnswer
    Titanium ⭐️⭐️⭐️
    July 7, 2023

    @SandyClouds 
    What i am saying is that rest-api call response you got via command line is probably not the exact same request made by NiFi.   What that response json tells you is that the user that made the request only has READ to all three buckets. You did not share the complete rest-api call you made, so I am guessing you did not pass any user authentication token or clientAuth certificate in yoru rest-api request and this that request was treated as bing made by an anonymous user.  Since your bucket(s) have the "make publicly visible" checked, anonymous user will have only READ access to the buckets.

    In NiFi you are trying to start version control on a Process group.  That means that the NiFi node certificate needs to be authorized on all buckets and authorized to proxy the NiFi authenticated user (who also needs to have read, write on the specific bucket to start version control.  If the response received back to NiFi during the request is that nodes or user on has READ, then no buckets would be shown in that UI since the action that UI is performing would be a WRITE.  

    You should use the "developer tools" in your web browser to capture the actual full request  being made by NiFi when you try to start version control on a process group.  Most developer tolls even give you and option to "copy as curl" which you can then just paste in a cmd window and execute manually yourself outside of NiFi to see the same response NiFi is getting.

    My guess here is that you are having a TLS handshake issue resulting in yoru requests to NiFi-Registry being anonymous requests.   The NiFi logs are not going to show you details on what authorization was on the NiFi-Registry side (you'll need to check the nifi-registry-app.log for that).  All the NiFi log you shared shows is that your "new_admin" was authorized to execute the NiFi rest-api endpoint.  This triggering the connection attempt with NiFi-Registry which also appeared successful (likely as anonymous).

    If you look at the verbose output for the NiFi keystore, NiFi truststore, NiFi-Registry keystore, and NiFi-Registry truststore, you'll be able to determine if a mutual TLS exchange would even be possible between the two services using the keystore and truststores you have in place.  These versbose outputs which you can get using the keytool command will also show you the EKUs and SANs present on your certificates.

    <path to JDK>/keytool -v -list -keystore <keystore or truststore file>

    When prompted for password, you can just hit enter without providing password.

    The screenshots you shared showing the authorizations configured in NiFi-Registry are sufficient.  I was simply pointing out that you gave too much authorization to your NiFi node in the global policy section.   This is why i am leaning more to a TLS handshake issue resulting in an anonymous user authentication rather then an authorization issue.  You'll also want to inspect your nifi-registry-app.log for the authorization request coming from NiFi to see if is authorizing "anonymous" or some other string.  Perhaps you have some identity mapping pattern configured on NiFi-Registry side that is trimming out the CN  from your certificate DNs?  In that case you could also see this behavior because your NiFi-Registry would be looking up authorization on different identity string (instead of "CN=new_admin, OU=Nifi, i maybe looking for just "new_admin").  NiFi and NiFi-Registry authorizations are exact string matches (case sensitive); otherwise, treated as different users.

    If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

    Thank you,

    Matt

    Translate
      Terms & ConditionsAccessibility statement