Support Questions

sydney- · ‎05-22-2025

My team and I are using Apache Nifi to sync data between databases. Currently we are working through attempting to connect both out Production Nifi service and our Development Nifi service to our Nifi registry service.

The production Nifi service and the Nifi registry are running on one EC2 server and the development Nifi service is running on another. We have them all set up to be secure using LDAP and users/groups are managed in Windows server AD. We are having some issues trying to get the Nifi services connected to the Registry service and were hoping we could get some feedback or guidance on what we may be missing or doing incorrectly.

The Nifi production instance is available from the host https://nifi.example.com/nifi and the Nifi registry is available at https://nifi-registry.example.com/nifi-registry. We have followed the documentation and set the registry in Nifi to the correct endpoint, but are getting the following error:

"Unable to obtain listing of buckets: javax.net.ssl.SSLHandshakeException:PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target"

It appears none of our users, including the intial user and administrators, are able to add a user in the Nifi registry UI so we are attempting to do so through the configuration .xml files on the server. I will add what we have attempted thus far to try to connect the Nifi instances to our registry.

Added both the Nifi certificates to the registries truststore, and added the registry certificates to both Nifi services truststores.
Restarted all services, but still got the above error.
Added the development Nifi service as a user with the same permissions as our nifi-admin users in the authorizations.xml, users.xml, and authorizers.xml.
We generated a UUID for the user and added <property name="NiFi Identity 1">cn=dev-nifi,ou=servers,dc=example,dc=com</property> to the access policy provider with the identifier 'file-access-policy-provider'. We then added the users generated uuid to the authorizations.xml to give permission to be able to access the buckets
We attempted to find which policy would allow the Nifi user to proxy user requests as well, and settled for 'proxy'.

We restarted the instance and checked to see if the user was created in the Registry UI, but it was not. Checked our dev-nifi UI and attempted to start version control. We did not get the above error, but no buckets are being found, despite there being buckets in the registry.

We’ve reviewed the official documentation but have not found a resolution. We appreciate any time and help you can offer.

MattWho · ‎05-29-2025

@lynott

I see some issues in your configurations. And have some questions in order to address some other potential issues.

Authorizers.xml:

File-user-group-provider

<identifier>file-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>
        <property name="Users File">./conf/users.xml</property>
        <property name="Initial User Identity 1">employee</property>
    </userGroupProvider>

Here you are defining manually a "user identity" of "employee". Is this "employee" a user that is authenticating into your NiFi via ldap authentication. If so, I'd expect this user identity to be returned by the ldap-user-group-provider instead of the file-user-group-provider. You can NOT have two user-group-provider return the same user identity. So if "employee" is going to be returned by ldap-user-group-provider, it needs to be removed from file-user-group-provider.

NOTE: The file-user-group-provider will only generate the users.xml if it does not already exist. So modifications to the configuration in the file-user-group-provider will NOT result in any changes to an already existing users.xml. the users.xml file will only contain manually added user and group identities. user and group identities returned by other user-group-provider will not get added to the users.xml.

ldap-user-group-provider
I assume you ldap connection is good, so I'll just focus on the user sync and group sync areas:

        <property name="User Search Base"></property>
        <property name="User Object Class"></property>
        <property name="User Search Scope">SUBTREE</property>
        <property name="User Search Filter"></property>
        <property name="User Identity Attribute">cn</property>
        <property name="User Group Name Attribute"></property>
        <property name="User Group Name Attribute - Referenced Group Attribute"></property>

        <property name="Group Search Base">OU=COMPANYGroups,DC=corp,DC=company,DC=com</property>
        <property name="Group Object Class">group</property>
        <property name="Group Search Scope">ONE_LEVEL</property>
        <property name="Group Search Filter">((cn=nifi-admins))</property>
        <property name="Group Name Attribute">cn</property>
        <property name="Group Member Attribute">member</property>
        <property name="Group Member Attribute - Referenced User Attribute"></property>

As configured you are performing a group sync that will return all the "member" attribute values from the "nifi-admins" group.
1. Are the member attribute values full user DNs?
Because you have "cn" configured in the user sync section "user Identity Attribute". each returned "member" is going to be looked up in order to get the cn value.
Since NiFi and NiFi-registry user and group identities are case sensitive, it is important that the user identities authorized match exactly with the user identity returned by user authentication in to NiFi-Registry. I see from the nifi-registry.properties file that you are using the ldap-provider for user authentication. Assuming you a re logging in using the user's "cn" as the username, you need to make sure that "USE_USERNAME" is being used and not "USE_DN" in that ldap-provider.

Composite-provider:

    <userGroupProvider>
        <identifier>composite-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.CompositeUserGroupProvider</class>
        <property name="Configurable User Group Provider">file-user-group-provider</property>
        <property name="User Group Provider 1">ldap-user-group-provider</property>
    </userGroupProvider>

Here you are using the "composite-user-group-provider", but are trying to use a configurable provider with it. Instead you should be using:

 <userGroupProvider>
        <identifier>composite-configurable-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider</class>
        <property name="Configurable User Group Provider">file-user-group-provider</property>
        <property name="User Group Provider 1">ldap-user-group-provider</property>
    </userGroupProvider>

You'll notice the "class" is different between these two providers. The "identifier" is not important, but must match the identifier set in your file-access-policy-provider.

file-access-policy-prpovider

<identifier>file-access-policy-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>
        <property name="User Group Provider">composite-user-group-provider</property>
        <property name="Authorizations File">./conf/authorizations.xml</property>
        <property name="Initial Admin Identity">employee</property>
        <property name="NiFi Group Name"></property>

        <!--<property name="NiFi Identity 1"></property>-->
    </accessPolicyProvider

The file-access-policy provider uses the authorizations.xml file to store all configured authorizations. It will only generate a authorizations.xml file if one does not already exist. It can be configured with an "initial Admin Identity". This configured Initial Admin Identity must be returned by one of yoru user-group-providers. In your case "employee" is being used here. The "NiFi Identity 1", "NiFi Identity 2", "NiFi Identity <xyz>" can be added to define your NiFi node certificate Identities which would allow this provider to seed the proxy policies needed by the NiFi hosts. But keep in mind that the file-user-group-provider would also need to generate these identities for this to work.

-------------------

users.xml and authorizations.xml files:
Your users.xml only contains one entry for "employee" which corresponds with your authorizers configuration. That user is assigned a UUID. I can then see that UUID properly authorized as and admin user in the authorizations.xml file.

What I do not see in the users.xml file is any group defined with uuid:

8c295cae-a773-4d6a-98cd-eef47d0b8189

2. I assume this is the uuid for your ldap-user-group-provider returned "nifi-admins" group?
3. I assume you manually added this group to the authorizations.xml file since you mentioned you have been unable to manually configure user and policies from within the NiFi-Registry UI yet?

----------------------
NiFi-Registry UI:

4. When you login to the NiFi-Registry using user "employee" do you not see "employee" displayed in upper right corner of UI along with the wrench icon?

Remember that only case sensitive "employee" and what ever group happens to be "8c295cae-a773-4d6a-98cd-eef47d0b8189" are currently authorized as an admins.

5. I assume you obtained "8c295cae-a773-4d6a-98cd-eef47d0b8189" by enabling debug in the logback.xml for the ldap-user-group-provider class? Or is this the uuid for the NiFi node certificate derived user identity?

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

View solution in original post

lynott · ‎06-03-2025

My apologizes, the only thing that seemed to change was that we are able to add users in the UI now.
The question is how to get the correct Nifi node certificate Identities for both our dev-nifi and our production-nifi to set as users? when I run openssl s_client -connect dev-nifi.example.com:443 -showcerts I get a wildcard CN for both
*.example.com before the leaf certificate in the chain. So, I feel like we might be missing something in the setup, or we have it configured wrong.

Is the registry expecting something else for the nifi node?

We are still unable to load buckets from both; I am logging into the Nifi instances as my group user 'lynott'

MattWho · ‎06-03-2025

@lynott
It sounds like you created wildcard certificates toi use with your NiFi and NiFi-Registry services/instances. I strongly discourage this from a security standpoint. NiFi utilizes its certificates to perform both clientAuth and ServerAuth. When used to perform clientAuth, such as connect to NiFi-Registry, the clientAuth CN is presented as the client/user identifier. This means the server side (NiFi-Registry in this use case) would need to authorize that wildcard CN which exposes your NiFi-Registry to unauthorized access by any clientAuth certificate that matches against that wildcard authorized entity (*.example.com).

While I recommend and encourage creation off unique certificates per node with a shared common name as one of the SAN entries as a security best practice. It is possible to create once certificate that contains SAN entries for every host on which hat certificate is used. However, the CN in that one certificate should not use wildcards, so that authorizations can not resolve to any other value then the expected non wildcard CN.

Looking back at your nifi-registry.properties file, I can se that you have not https://nifi.apache.org/nifi-docs/administration-guide.html#identity-mapping-properties configured.

nifi.registry.security.identity.mapping.pattern.<somestring>=
nifi.registry.security.identity.mapping.value.<somestring>=
nifi.registry.security.identity.mapping.transfrom.<somestring>=

Identity Mapping Properties are used to manipulated user/client identities post authentication and pre-authorization. Without using identity.mapping.patterns, the complete clientAuth DistinquishedName (DN) will be passed to the authorizer in NiFi where it is expected to be authorized to Can Proxy Request (read, write, delete) and Can Manage Buckets (Read).

Since you aer are using a wildcard DN, authorizing the NiFi node hostnames in NiFi-Registry is going to accomplish nothing since those hostnames can't be extarcted from the DN for authorization.

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Cloudera Community

Support Questions

Unable to connect NiFi to NiFi Registry