Skip to main content
sydney-
sydney-
Bronze ⭐️⭐️⭐️
May 22, 2025
Solved

Unable to connect NiFi to NiFi Registry

My team and I are using Apache Nifi to sync data between databases. Currently we are working through attempting to connect both out Production Nifi service and our Development Nifi service to our Nifi registry service.

The production Nifi service and the Nifi registry are running on one EC2 server and the development Nifi service is running on another. We have them all set up to be secure using LDAP and users/groups are managed in Windows server AD. We are having some issues trying to get the Nifi services connected to the Registry service and were hoping we could get some feedback or guidance on what we may be missing or doing incorrectly.

The Nifi production instance is available from the host https://nifi.example.com/nifi and the Nifi registry is available at https://nifi-registry.example.com/nifi-registry. We have followed the documentation and set the registry in Nifi to the correct endpoint, but are getting the following error:

"Unable to obtain listing of buckets: javax.net.ssl.SSLHandshakeException:PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target"

It appears none of our users, including the intial user and administrators, are able to add a user in the Nifi registry UI so we are attempting to do so through the configuration .xml files on the server. I will add what we have attempted thus far to try to connect the Nifi instances to our registry.

  • Added both the Nifi certificates to the registries truststore, and added the registry certificates to both Nifi services truststores.
  • Restarted all services, but still got the above error.
  • Added the development Nifi service as a user with the same permissions as our nifi-admin users in the authorizations.xml, users.xml, and authorizers.xml.
  • We generated a UUID for the user and added <property name="NiFi Identity 1">cn=dev-nifi,ou=servers,dc=example,dc=com</property> to the access policy provider with the identifier 'file-access-policy-provider'. We then added the users generated uuid to the authorizations.xml to give permission to be able to access the buckets
  • We attempted to find which policy would allow the Nifi user to proxy user requests as well, and settled for 'proxy'.

We restarted the instance and checked to see if the user was created in the Registry UI, but it was not. Checked our dev-nifi UI and attempted to start version control. We did not get the above error, but no buckets are being found, despite there being buckets in the registry.

We’ve reviewed the official documentation but have not found a resolution. We appreciate any time and help you can offer.

MattWho
MattWhoAnswer
Titanium ⭐️⭐️⭐️
May 29, 2025

@lynott 

I see some issues in your configurations. And have some questions in order to address some other potential issues.

Authorizers.xml:

  • File-user-group-provider
<identifier>file-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>
        <property name="Users File">./conf/users.xml</property>
        <property name="Initial User Identity 1">employee</property>
    </userGroupProvider>​

Here you are defining manually a "user identity" of "employee".    Is this "employee" a user that is authenticating into your NiFi via ldap authentication.  If so, I'd expect this user identity to be returned by the ldap-user-group-provider instead of the file-user-group-provider.  You can NOT have two user-group-provider return the same user identity.  So if "employee" is going to be returned by ldap-user-group-provider, it needs to be removed from file-user-group-provider.

NOTE: The file-user-group-provider will only generate the users.xml if it does not already exist.  So modifications to the configuration in the file-user-group-provider will NOT result in any changes to an already existing users.xml.  the users.xml file will only contain manually added user and group identities. user and group identities returned by other user-group-provider will not get added to the users.xml.

  • ldap-user-group-provider
    I assume you ldap connection is good, so I'll just focus on the user sync and group sync areas:
        <property name="User Search Base"></property>
        <property name="User Object Class"></property>
        <property name="User Search Scope">SUBTREE</property>
        <property name="User Search Filter"></property>
        <property name="User Identity Attribute">cn</property>
        <property name="User Group Name Attribute"></property>
        <property name="User Group Name Attribute - Referenced Group Attribute"></property>

        <property name="Group Search Base">OU=COMPANYGroups,DC=corp,DC=company,DC=com</property>
        <property name="Group Object Class">group</property>
        <property name="Group Search Scope">ONE_LEVEL</property>
        <property name="Group Search Filter">((cn=nifi-admins))</property>
        <property name="Group Name Attribute">cn</property>
        <property name="Group Member Attribute">member</property>
        <property name="Group Member Attribute - Referenced User Attribute"></property>​

As configured you are performing a group sync that will return all the "member" attribute values from the "nifi-admins" group.  
1. Are the member attribute values full user DNs?  
Because you have "cn" configured in the user sync section "user Identity Attribute". each returned "member" is going to be looked up in order to get the cn value.
Since NiFi and NiFi-registry user and group identities are case sensitive, it is important that the user identities authorized match exactly with the user identity returned by user authentication in to NiFi-Registry.  I see from the nifi-registry.properties file that you are using the ldap-provider for user authentication.  Assuming you a re logging in using the user's "cn" as the username, you need to make sure that "USE_USERNAME" is being used and not "USE_DN" in that ldap-provider.

  • Composite-provider:

 

    <userGroupProvider>
        <identifier>composite-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.CompositeUserGroupProvider</class>
        <property name="Configurable User Group Provider">file-user-group-provider</property>
        <property name="User Group Provider 1">ldap-user-group-provider</property>
    </userGroupProvider>

 

Here you are using the "composite-user-group-provider", but are trying to use a configurable provider with it.  Instead you should be using:

 

 <userGroupProvider>
        <identifier>composite-configurable-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider</class>
        <property name="Configurable User Group Provider">file-user-group-provider</property>
        <property name="User Group Provider 1">ldap-user-group-provider</property>
    </userGroupProvider>

 

 You'll notice the "class" is different between these two providers.  The "identifier" is not important, but must match the identifier set in your file-access-policy-provider.

  • file-access-policy-prpovider
<identifier>file-access-policy-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>
        <property name="User Group Provider">composite-user-group-provider</property>
        <property name="Authorizations File">./conf/authorizations.xml</property>
        <property name="Initial Admin Identity">employee</property>
        <property name="NiFi Group Name"></property>

        <!--<property name="NiFi Identity 1"></property>-->
    </accessPolicyProvider​

The file-access-policy provider uses the authorizations.xml file to store all configured authorizations.  It will only generate a authorizations.xml file if one does not already exist.  It can be configured with an "initial Admin Identity".  This configured Initial Admin Identity must be returned by one of yoru user-group-providers.  In your case "employee" is being used here.  The "NiFi Identity 1", "NiFi Identity 2", "NiFi Identity <xyz>" can be added to define your NiFi node certificate Identities which would allow this provider to seed the proxy policies needed by the NiFi hosts.  But keep in mind that the file-user-group-provider would also need to generate these identities for this to work. 

-------------------

users.xml and authorizations.xml files:
Your users.xml only contains one entry for "employee" which corresponds with your authorizers configuration.  That user is assigned a UUID.  I can then see that UUID properly authorized as and admin user in the authorizations.xml file.

What I do not see in the users.xml file is any group defined with uuid:

 

8c295cae-a773-4d6a-98cd-eef47d0b8189

 

2. I assume this is the uuid for your ldap-user-group-provider returned "nifi-admins" group?   
3. I assume you manually added this group to the authorizations.xml file since you mentioned you have been unable to manually configure user and policies from within the NiFi-Registry UI yet?

----------------------
NiFi-Registry UI:

4. When you login to the NiFi-Registry using user "employee" do you not see "employee" displayed in upper right corner of UI  along with the wrench icon?

MattWho_0-1748546046110.png

Remember that only case sensitive "employee" and what ever group happens to be "8c295cae-a773-4d6a-98cd-eef47d0b8189" are currently authorized as an admins.

 5. I assume you obtained "8c295cae-a773-4d6a-98cd-eef47d0b8189" by enabling debug in the logback.xml for the ldap-user-group-provider class?  Or is this the uuid for the NiFi node certificate derived user identity?

 

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

View in thread

11 replies

VidyaSargur
Community Manager
VidyaSargur
Community Manager
May 23, 2025

@sydney-, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @MattWho  @Shelton , who may be able to assist you further.

Please feel free to provide any additional information or details about your query. We hope that you will find a satisfactory solution to your question.

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
Shelton
Shelton
Gold ⭐️⭐️⭐️
June 5, 2025

@sydney- 
The SSL handshake error you're encountering is a common issue when connecting NiFi instances to NiFi Registry in secure environments it indicates that your NiFi instances cannot verify the SSL certificate presented by the NiFi Registry server.

javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider. certpath.SunCertPathBuilder
Exception: 
unable to find valid certification path to requested target

Based on your description, there are several areas to address.

  1. The certificate used by NiFi Registry is self-signed or not issued by a trusted Certificate Authority (CA)
  2. The certificate chain is incomplete
  3. The truststore configuration is incorrect

1. Certificate Trust Configuration
Verify Certificate Chain:

# Check if certificate is in NiFi truststore (repeat for each instance)
keytool -list -v -keystore /path/to/nifi/truststore.jks -storepass [password]
# Check if certificate is in Registry truststore
keytool -list -v -keystore /path/to/registry/truststore.jks -storepass [password]

# Verify the Registry's certificate chain
openssl s_client -connect nifi-registry.example.com:443 -showcerts

Ensure Complete Certificate Chain:

  • Add the Registry's complete certificate chain (including intermediate CAs) to NiFi's truststore
  • Add NiFi's complete certificate chain to the Registry's truststore
# Add Registry certificate to NiFi truststore
keytool -import -alias nifi-registry -file registry-cert.pem -keystore /path/to/nifi/conf/truststore.jks -storepass [password]

# Add NiFi certificate to Registry truststore
keytool -import -alias nifi-prod -file nifi-cert.pem -keystore /path/to/registry/conf/truststore.jks -storepass [password]

2. Proper Certificate Exchange
Ensure you've exchanged certificates correctly export NiFi Registry's public certificate

keytool -exportcert -alias nifi-registry -keystore /path/to/registry/keystore.jks -file registry.crt -storepass [password]

Import this certificate into each NiFi instance's truststore

keytool -importcert -alias nifi-registry -keystore /path/to/nifi/truststore.jks -file registry.crt -storepass [password] -noprompt

3. NiFi Registry Connection Configuration
In your NiFi instance (nifi.properties), verify

# Registry client properties
nifi.registry.client.name=NiFi Registry
nifi.registry.client.url=https://nifi-registry.example.com/nifi-registry
nifi.registry.client.timeout.connect=30 secs
nifi.registry.client.timeout.read=30 secs

Verify these configuration files in NiFi (production/development)

# nifi.properties:
nifi.registry.client.ssl.protocol=TLS
nifi.registry.client.truststore.path=/path/to/truststore.jks
nifi.registry.client.truststore.password=[password]
nifi.registry.client.truststore.type=JKS

In NiFi Registry

# nifi-registry.properties:
nifi.registry.security.truststore.path=/path/to/truststore.jks
nifi.registry.security.truststore.password=[password]
nifi.registry.security.truststore.type=JKS

4. LDAP Configuration
For your LDAP integration issues in authorizers.xml ensure you have

<accessPolicyProvider>
    <identifier>file-access-policy-provider</identifier>
   <class>org.apache.nifi.registry.security.authorization.FileAccessPolicyProvider</class>
    <property name="User Group Provider">ldap-user-group-provider</property>
    <property name="Authorizations File">./conf/authorizations.xml</property>
    <property name="Initial Admin Identity">cn=admin-user,ou=users,dc=example,dc=com</property>
    <property name="NiFi Identity 1">cn=dev-nifi,ou=servers,dc=example,dc=com</property>
</accessPolicyProvider>

In the authorizations.xml add appropriate policies for the dev-nifi identity

<policy identifier="some-uuid" resource="/buckets" action="READ">
    <user identifier="dev-nifi-uuid"/>
</policy>

5. Proxy Configuration
For proxy user requests, add in nifi.properties

nifi.registry.client.proxy.identity=cn=dev-nifi,ou=servers,dc=example,dc=com

6. Restart Order
After making changes restart the Nifi instance in the below order

  1. NiFi Registry first
  2. Then restart all NiFi instances


Happy hadoping

Translate
MattWho
Titanium ⭐️⭐️⭐️
MattWho
Titanium ⭐️⭐️⭐️
May 28, 2025

@sydney- 
The exception you shared:
"Unable to obtain listing of buckets: javax.net.ssl.SSLHandshakeException:PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target"

Indicates a TLS trust issue within the mutualTLS handshake between your NiFi and NiFi-Registry.  Perhaps the complete trust chain is missing in the truststores.

Looking at the output from OpenSSL may be able to help you see what is missing.

openssl s_client -connect <nifi-regsitry-hostname>:<nifi-registry-port> -showcerts

openssl s_client -connect <nifi-hostname>:<nifi-port> -showcerts

  Need to get past this issue before dealing with and proxied user identity issues.

 

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Translate
lynott
lynott
Silver ⭐️⭐️
May 28, 2025

Hello Matt, thank you for your response. I had followed the steps here Re: NIfi and Nifi Registry Integration - Cloudera Community that you posted previously.

openssl s_client -connect nifi.example.com:443 -showcerts

# copied the certificate chain into nifi-1.28.0.chain.crt as a nifi-cert already existed in the registries truststore

keytool -import -alias nifi-1.28-crt -keystore conf/truststore.p12 -file nifi-1.28.chain.crt -storepass $STORE_PASS

systemctl restart nifi-registry.service

The error referenced in the above post had already been resolved, just wanted to share what all we have done and tried.

 The main issue now is that even though we have buckets in the registry, when we click to start version control on a processor group it says there are no buckets available. We also have the issue of no nifi-registry users being able to add a user for the nifi instances. 

Translate
MattWho
Titanium ⭐️⭐️⭐️
MattWho
Titanium ⭐️⭐️⭐️
May 28, 2025

@lynott 
When trying to start version control, the NiFi instance will authenticate with NiFi-registry via a MutualTLS exchange. The DN from NiFi's keytore clientAuth privateKey entry is used as the user identity (post NiFi-Registry applying any matching identity.mapping patterns to it).  In the request for list of buckets will be the "user identity" displayed in teh upper right corner of NiFi when the request was made.  The NiFi node is proxying the request on behalf of that user identity.   So If you are not seeing any buckets, either there is a proxy issue for one of your NiFi nodes or the NiFi user identity is not properly authorized in NiFi-Registry for any buckets.  

Without seeing your NiFi-registry configurations, it would be difficult to say exactly what your issue is.

This includes looking at:
NiFi-Registry.properties
authorizers.xml
users.xml
authorizations.xml

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt





Translate
lynott
lynott
Silver ⭐️⭐️
June 2, 2025

@MattWho 

  Thank you for your quick response, I have read over the above and will do my best to answer your questions, as the initial set up was done prior to me taking on the project. I have Updated the Authorizer.xml and removed 'employee' as the initial admin from the file-user-group-provider

In the identity-providers.xml <property name="Identity Strategy">USE_USERNAME</property> was already set correctly

I changed the class in the authorizers.xml for the composite-user-group-provider identity to org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider and made sure the file-access-policy-provider used the correct User group provider

the UUID in the authorizations.xml were already there when I began taking over the work on this project and I am going to assume they were added when the registry service was first started.
When accessing the Registry UI I do indeed see my user in the upper right hand corner.
Following these steps provided members of the Nifi admin group to be able to add a user in the UI.
I think where I may be going wrong here is the Nifi node certificate Identities. I added <property name="Nifi identity 1">CN=dev-nifi.example.com</property> to both the file-user-group-provider and the file-access-policy-provider. I think this may be an issue with how I am setting the Identity here perhaps, I also tried in the UI. I do now see three UUIDs in the Authorization.xml/users.xml after restarting the Nifi-registry.service on the EC2 instance; one for the 'employee' one for the group and one I am assuming is for the Nifi identity. 

Translate
MattWho
Titanium ⭐️⭐️⭐️
MattWho
Titanium ⭐️⭐️⭐️
June 3, 2025

@lynott @sydney- 

Are you good now?  Your last response contains no questions.
If my previous response(s) helped with your initial issue, please a take a moment to click accepted on those responses.

Thank you,
Matt

Translate
lynott
lynott
Silver ⭐️⭐️
June 3, 2025

My apologizes, the only thing that seemed to change was that we are able to add users in the UI now. 
The question is how to get the correct  Nifi node certificate Identities for both our dev-nifi and our production-nifi to set as users? when I run openssl s_client -connect dev-nifi.example.com:443 -showcerts I get a wildcard CN for both 
*.example.com before the leaf certificate in the chain. So, I feel like we might be missing something in the setup, or we have it configured wrong.

Is the registry expecting something else for the nifi node?

We are still unable to load buckets from both; I am logging into the Nifi instances as my group user 'lynott' 

Translate
MattWho
Titanium ⭐️⭐️⭐️
MattWho
Titanium ⭐️⭐️⭐️
June 3, 2025

@lynott 
It sounds like you created wildcard certificates toi use with your NiFi and NiFi-Registry  services/instances.  I strongly discourage this from a security standpoint.  NiFi utilizes its certificates to perform both clientAuth and ServerAuth. When used to perform clientAuth, such as connect to NiFi-Registry, the clientAuth CN is presented as the client/user identifier.  This means the server side (NiFi-Registry in this use case) would need to authorize that wildcard CN which exposes your NiFi-Registry to unauthorized access by any clientAuth certificate that matches against that wildcard authorized entity  (*.example.com).    

While I recommend and encourage creation off unique certificates per node with a shared common name as one of the SAN entries as a security best practice. It is possible to create once certificate that contains SAN entries for every host on which hat certificate is used.  However, the CN in that one certificate should not use wildcards, so that authorizations can not resolve to any other value then the expected  non wildcard CN.  

Looking back at your nifi-registry.properties file, I can se that you have not https://nifi.apache.org/nifi-docs/administration-guide.html#identity-mapping-properties configured.

 

nifi.registry.security.identity.mapping.pattern.<somestring>=
nifi.registry.security.identity.mapping.value.<somestring>=
nifi.registry.security.identity.mapping.transfrom.<somestring>=

 

Identity Mapping Properties are used to manipulated user/client identities post authentication and pre-authorization.  Without using identity.mapping.patterns, the complete clientAuth DistinquishedName (DN) will be passed to the authorizer in NiFi where it is expected to be authorized to Can Proxy Request (read, write, delete) and Can Manage Buckets (Read).

Since you aer are using a wildcard DN, authorizing the NiFi node hostnames in NiFi-Registry is going to accomplish nothing since those hostnames can't be extarcted from the DN for authorization.

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Translate
Terms & ConditionsAccessibility statement