Unable to connect NiFi to NiFi Registry

[#1]

My team and I are using Apache Nifi to sync data between databases. Currently we are working through attempting to connect both out Production Nifi service and our Development Nifi service to our Nifi registry service.

The production Nifi service and the Nifi registry are running on one EC2 server and the development Nifi service is running on another. We have them all set up to be secure using LDAP and users/groups are managed in Windows server AD. We are having some issues trying to get the Nifi services connected to the Registry service and were hoping we could get some feedback or guidance on what we may be missing or doing incorrectly.

The Nifi production instance is available from the host https://nifi.example.com/nifi and the Nifi registry is available at https://nifi-registry.example.com/nifi-registry. We have followed the documentation and set the registry in Nifi to the correct endpoint, but are getting the following error:

"Unable to obtain listing of buckets: javax.net.ssl.SSLHandshakeException:PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target"

It appears none of our users, including the intial user and administrators, are able to add a user in the Nifi registry UI so we are attempting to do so through the configuration .xml files on the server. I will add what we have attempted thus far to try to connect the Nifi instances to our registry.

• Added both the Nifi certificates to the registries truststore, and added the registry certificates to both Nifi services truststores.
• Restarted all services, but still got the above error.
• Added the development Nifi service as a user with the same permissions as our nifi-admin users in the authorizations.xml, users.xml, and authorizers.xml.
• We generated a UUID for the user and added <property name="NiFi Identity 1">cn=dev-nifi,ou=servers,dc=example,dc=com</property> to the access policy provider with the identifier 'file-access-policy-provider'. We then added the users generated uuid to the authorizations.xml to give permission to be able to access the buckets
• We attempted to find which policy would allow the Nifi user to proxy user requests as well, and settled for 'proxy'.

We restarted the instance and checked to see if the user was created in the Registry UI, but it was not. Checked our dev-nifi UI and attempted to start version control. We did not get the above error, but no buckets are being found, despite there being buckets in the registry.

We’ve reviewed the official documentation but have not found a resolution. We appreciate any time and help you can offer.

[#2]

@sydney-, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @MattWho  @Shelton , who may be able to assist you further.

Please feel free to provide any additional information or details about your query. We hope that you will find a satisfactory solution to your question.

[#3]

@sydney- 
The SSL handshake error you're encountering is a common issue when connecting NiFi instances to NiFi Registry in secure environments it indicates that your NiFi instances cannot verify the SSL certificate presented by the NiFi Registry server.

javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider. certpath.SunCertPathBuilder
Exception: 
unable to find valid certification path to requested target

Based on your description, there are several areas to address.

1. The certificate used by NiFi Registry is self-signed or not issued by a trusted Certificate Authority (CA)
2. The certificate chain is incomplete
3. The truststore configuration is incorrect

1. Certificate Trust Configuration
Verify Certificate Chain:

# Check if certificate is in NiFi truststore (repeat for each instance)
keytool -list -v -keystore /path/to/nifi/truststore.jks -storepass [password]
# Check if certificate is in Registry truststore
keytool -list -v -keystore /path/to/registry/truststore.jks -storepass [password]

# Verify the Registry's certificate chain
openssl s_client -connect nifi-registry.example.com:443 -showcerts

Ensure Complete Certificate Chain:

• Add the Registry's complete certificate chain (including intermediate CAs) to NiFi's truststore
• Add NiFi's complete certificate chain to the Registry's truststore

# Add Registry certificate to NiFi truststore
keytool -import -alias nifi-registry -file registry-cert.pem -keystore /path/to/nifi/conf/truststore.jks -storepass [password]

# Add NiFi certificate to Registry truststore
keytool -import -alias nifi-prod -file nifi-cert.pem -keystore /path/to/registry/conf/truststore.jks -storepass [password]

2. Proper Certificate Exchange
Ensure you've exchanged certificates correctly export NiFi Registry's public certificate

keytool -exportcert -alias nifi-registry -keystore /path/to/registry/keystore.jks -file registry.crt -storepass [password]

Import this certificate into each NiFi instance's truststore

keytool -importcert -alias nifi-registry -keystore /path/to/nifi/truststore.jks -file registry.crt -storepass [password] -noprompt

3. NiFi Registry Connection Configuration
In your NiFi instance (nifi.properties), verify

# Registry client properties
nifi.registry.client.name=NiFi Registry
nifi.registry.client.url=https://nifi-registry.example.com/nifi-registry
nifi.registry.client.timeout.connect=30 secs
nifi.registry.client.timeout.read=30 secs

Verify these configuration files in NiFi (production/development)

# nifi.properties:
nifi.registry.client.ssl.protocol=TLS
nifi.registry.client.truststore.path=/path/to/truststore.jks
nifi.registry.client.truststore.password=[password]
nifi.registry.client.truststore.type=JKS

In NiFi Registry

# nifi-registry.properties:
nifi.registry.security.truststore.path=/path/to/truststore.jks
nifi.registry.security.truststore.password=[password]
nifi.registry.security.truststore.type=JKS

4. LDAP Configuration
For your LDAP integration issues in authorizers.xml ensure you have

<accessPolicyProvider>
    <identifier>file-access-policy-provider</identifier>
   <class>org.apache.nifi.registry.security.authorization.FileAccessPolicyProvider</class>
    <property name="User Group Provider">ldap-user-group-provider</property>
    <property name="Authorizations File">./conf/authorizations.xml</property>
    <property name="Initial Admin Identity">cn=admin-user,ou=users,dc=example,dc=com</property>
    <property name="NiFi Identity 1">cn=dev-nifi,ou=servers,dc=example,dc=com</property>
</accessPolicyProvider>

In the authorizations.xml add appropriate policies for the dev-nifi identity

<policy identifier="some-uuid" resource="/buckets" action="READ">
    <user identifier="dev-nifi-uuid"/>
</policy>

5. Proxy Configuration
For proxy user requests, add in nifi.properties

nifi.registry.client.proxy.identity=cn=dev-nifi,ou=servers,dc=example,dc=com

6. Restart Order
After making changes restart the Nifi instance in the below order

1. NiFi Registry first
2. Then restart all NiFi instances

Happy hadoping

[#4]

@sydney- 
The exception you shared:
"Unable to obtain listing of buckets: javax.net.ssl.SSLHandshakeException:PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target"

Indicates a TLS trust issue within the mutualTLS handshake between your NiFi and NiFi-Registry.  Perhaps the complete trust chain is missing in the truststores.

Looking at the output from OpenSSL may be able to help you see what is missing.

openssl s_client -connect <nifi-regsitry-hostname>:<nifi-registry-port> -showcerts

openssl s_client -connect <nifi-hostname>:<nifi-port> -showcerts

  Need to get past this issue before dealing with and proxied user identity issues.

 

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#5]

Hello Matt, thank you for your response. I had followed the steps here Re: NIfi and Nifi Registry Integration - Cloudera Community that you posted previously.

openssl s_client -connect nifi.example.com:443 -showcerts

# copied the certificate chain into nifi-1.28.0.chain.crt as a nifi-cert already existed in the registries truststore

keytool -import -alias nifi-1.28-crt -keystore conf/truststore.p12 -file nifi-1.28.chain.crt -storepass $STORE_PASS

systemctl restart nifi-registry.service

The error referenced in the above post had already been resolved, just wanted to share what all we have done and tried.

 The main issue now is that even though we have buckets in the registry, when we click to start version control on a processor group it says there are no buckets available. We also have the issue of no nifi-registry users being able to add a user for the nifi instances. 

[#6]

@lynott 
When trying to start version control, the NiFi instance will authenticate with NiFi-registry via a MutualTLS exchange. The DN from NiFi's keytore clientAuth privateKey entry is used as the user identity (post NiFi-Registry applying any matching identity.mapping patterns to it).  In the request for list of buckets will be the "user identity" displayed in teh upper right corner of NiFi when the request was made.  The NiFi node is proxying the request on behalf of that user identity.   So If you are not seeing any buckets, either there is a proxy issue for one of your NiFi nodes or the NiFi user identity is not properly authorized in NiFi-Registry for any buckets.  

Without seeing your NiFi-registry configurations, it would be difficult to say exactly what your issue is.

This includes looking at:
NiFi-Registry.properties
authorizers.xml
users.xml
authorizations.xml

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#7]

Adding those files with the sensitive information removed

Authorizers.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizers>
    <userGroupProvider>
        <identifier>file-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>
        <property name="Users File">./conf/users.xml</property>
        <property name="Initial User Identity 1">employee</property>
    </userGroupProvider>

    <userGroupProvider>
        <identifier>ldap-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.ldap.tenants.LdapUserGroupProvider</class>
        <property name="Authentication Strategy">LDAPS</property>

        <property name="Manager DN">CN=nifildap,OU=ServiceAccounts,OU=COMPANYUsers,DC=corp,DC=company,DC=com</property>
        <property name="Manager Password">MANAGER_PASS</property>

        <property name="TLS - Keystore">./conf/ldap_keystore.jks</property>
        <property name="TLS - Keystore Password">KEYSTORE_PASSWORD</property>
        <property name="TLS - Keystore Type">jks</property>
        <property name="TLS - Truststore">./conf/ldap_truststore.jks</property>
        <property name="TLS - Truststore Password">TRUSTSTORE_PASSWORD</property>
        <property name="TLS - Truststore Type">jks</property>
        <property name="TLS - Client Auth"></property>
        <property name="TLS - Protocol">TLSv1.2</property>
        <property name="TLS - Shutdown Gracefully"></property>

        <property name="Referral Strategy">FOLLOW</property>
        <property name="Connect Timeout">10 secs</property>
        <property name="Read Timeout">10 secs</property>

        <property name="Url">ldaps://dc1.corp.company.com:636</property>
        <property name="Page Size"></property>
        <property name="Sync Interval">30 mins</property>
        <property name="Group Membership - Enforce Case Sensitivity">false</property>

        <property name="User Search Base"></property>
        <property name="User Object Class"></property>
        <property name="User Search Scope">SUBTREE</property>
        <property name="User Search Filter"></property>
        <property name="User Identity Attribute">cn</property>
        <property name="User Group Name Attribute"></property>
        <property name="User Group Name Attribute - Referenced Group Attribute"></property>

        <property name="Group Search Base">OU=COMPANYGroups,DC=corp,DC=company,DC=com</property>
        <property name="Group Object Class">group</property>
        <property name="Group Search Scope">ONE_LEVEL</property>
        <property name="Group Search Filter">((cn=nifi-admins))</property>
        <property name="Group Name Attribute">cn</property>
        <property name="Group Member Attribute">member</property>
        <property name="Group Member Attribute - Referenced User Attribute"></property>
    </userGroupProvider>

    <userGroupProvider>
        <identifier>composite-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.CompositeUserGroupProvider</class>
        <property name="Configurable User Group Provider">file-user-group-provider</property>
        <property name="User Group Provider 1">ldap-user-group-provider</property>
    </userGroupProvider>

    <accessPolicyProvider>
        <identifier>file-access-policy-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>
        <property name="User Group Provider">composite-user-group-provider</property>
        <property name="Authorizations File">./conf/authorizations.xml</property>
        <property name="Initial Admin Identity">employee</property>
        <property name="NiFi Group Name"></property>

        <!--<property name="NiFi Identity 1"></property>-->
    </accessPolicyProvider>

    <authorizer>
        <identifier>managed-authorizer</identifier>
        <class>org.apache.nifi.registry.security.authorization.StandardManagedAuthorizer</class>
        <property name="Access Policy Provider">file-access-policy-provider</property>
    </authorizer>

</authorizers>

users.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
    <groups/>
    <users>
        <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309" identity="employee"/>
    </users>
</tenants>

Authorizations.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizations>
    <policies>
        <policy identifier="2dbc92a2-b091-3616-8e88-5078b9103b04" resource="/tenants" action="D">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="15e4e0bd-cb28-34fd-8587-f8d15162cba5" resource="/tenants" action="W">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="627410be-1717-35b4-a06f-e9362b89e0b7" resource="/tenants" action="R">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="2fd3fcf5-b10f-33fa-8d8e-b262fa34815e" resource="/actuator" action="R">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="01b87cb5-c0b6-342d-b108-d8bc03ab5cde" resource="/policies" action="D">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="cf4d8390-5ac7-3ff0-82ce-a274b5f88b21" resource="/swagger" action="W">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="0eaa47b9-e409-304e-8682-30d1b0d86d05" resource="/swagger" action="R">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="ff96062a-fa99-36dc-9942-0f6442ae7212" resource="/policies" action="R">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="ac587f43-6e1c-3890-81fd-83b4df2e678e" resource="/swagger" action="D">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="d59a54f7-6dd6-34ad-a279-a26ffdb9eef8" resource="/proxy" action="R">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="9d182b11-ebe3-3a7a-8731-98ce6d6e44fd" resource="/buckets" action="R">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="dfbf3c51-fdec-3328-b169-3b54eb033147" resource="/buckets" action="W">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="05b96464-9ec8-312a-8459-67812a8b48c1" resource="/buckets" action="D">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="2f470357-e82c-38ee-8062-ab6388d6ec75" resource="/actuator" action="W">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="3ee4703f-94ca-33c2-8060-17f5d313f560" resource="/actuator" action="D">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="287edf48-da72-359b-8f61-da5d4c45a270" resource="/proxy" action="W">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="ad99ea98-3af6-3561-ae27-5bf09e1d969d" resource="/policies" action="W">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="6dbdbffd-8a7d-32e1-ba3e-f600e6c69791" resource="/proxy" action="D">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="9db54cd5-07b7-49e4-9b22-010a0af52309"/>
        </policy>
        <policy identifier="ce2bf41b-96e7-497e-86bc-661e71066a25" resource="/buckets/9ebee5c0-88ef-435a-bce3-4ca448a4cc5a" action="D">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
        </policy>
        <policy identifier="d86fd825-1771-4935-9010-acc1f2d41a4c" resource="/buckets/9ebee5c0-88ef-435a-bce3-4ca448a4cc5a" action="R">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
        </policy>
        <policy identifier="9fe582c5-5bc5-4783-ac58-d30f3dccd85c" resource="/buckets/9ebee5c0-88ef-435a-bce3-4ca448a4cc5a" action="W">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
        </policy>
        <policy identifier="bc8e4c75-a893-4d85-892e-f0e6796f1abc" resource="/buckets/3eb5b85d-005b-43cd-8591-5d577bb31ca1" action="W">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="b495b01c-b1bc-33ec-ac0a-eb1f7fd572c8"/>
        </policy>
        <policy identifier="c192ea08-6584-44da-8400-354ab90649ed" resource="/buckets/3eb5b85d-005b-43cd-8591-5d577bb31ca1" action="R">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="b495b01c-b1bc-33ec-ac0a-eb1f7fd572c8"/>
        </policy>
        <policy identifier="cd42e4c9-212d-4fdf-9d0f-767cc2c20b89" resource="/buckets/3eb5b85d-005b-43cd-8591-5d577bb31ca1" action="D">
            <group identifier="8c295cae-a773-4d6a-98cd-eef47d0b8189"/>
            <user identifier="b495b01c-b1bc-33ec-ac0a-eb1f7fd572c8"/>
        </policy>
    </policies>
</authorizations>

nifi-registry.properies

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# web properties #
nifi.registry.web.war.directory=./lib
nifi.registry.web.http.host=
nifi.registry.web.http.port=
nifi.registry.web.https.host=0.0.0.0
nifi.registry.web.https.port=18443
nifi.registry.web.https.application.protocols=http/1.1
nifi.registry.web.jetty.working.directory=./work/jetty
nifi.registry.web.jetty.threads=200
nifi.registry.web.should.send.server.version=true

# security properties #
nifi.registry.security.keystore=./conf/keystore.p12
nifi.registry.security.keystoreType=PKCS12
nifi.registry.security.keystorePasswd=KEYSTORE_PASSWORD
nifi.registry.security.keyPasswd=KEY_PASSWORD
nifi.registry.security.truststore=./conf/truststore.p12
nifi.registry.security.truststoreType=PKCS12
nifi.registry.security.truststorePasswd=TRUSTSTORE_PASSWORD
nifi.registry.security.needClientAuth=false
nifi.registry.security.authorizers.configuration.file=./conf/authorizers.xml
nifi.registry.security.authorizer=managed-authorizer
nifi.registry.security.identity.providers.configuration.file=./conf/identity-providers.xml
nifi.registry.security.identity.provider=ldap-identity-provider

# sensitive property protection properties #
# nifi.registry.sensitive.props.additional.keys=

# providers properties #
nifi.registry.providers.configuration.file=./conf/providers.xml

# registry alias properties #
nifi.registry.registry.alias.configuration.file=./conf/registry-aliases.xml

# extensions working dir #
nifi.registry.extensions.working.directory=./work/extensions

# legacy database properties, used to migrate data from original DB to new DB below
# NOTE: Users upgrading from 0.1.0 should leave these populated, but new installs after 0.1.0 should leave these empty
nifi.registry.db.directory=
nifi.registry.db.url.append=

# database properties
nifi.registry.db.url=jdbc:h2:./database/nifi-registry-primary;AUTOCOMMIT=OFF;DB_CLOSE_ON_EXIT=FALSE;LOCK_MODE=3;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
nifi.registry.db.driver.class=org.h2.Driver
nifi.registry.db.driver.directory=
nifi.registry.db.username=nifireg
nifi.registry.db.password=nifireg
nifi.registry.db.maxConnections=5
nifi.registry.db.sql.debug=false

# extension directories #
# Each property beginning with "nifi.registry.extension.dir." will be treated as location for an extension,
# and a class loader will be created for each location, with the system class loader as the parent
#
#nifi.registry.extension.dir.1=/path/to/extension1
#nifi.registry.extension.dir.2=/path/to/extension2

nifi.registry.extension.dir.aws=./ext/aws/lib

# Identity Mapping Properties #
# These properties allow normalizing user identities such that identities coming from different identity providers
# (certificates, LDAP, Kerberos) can be treated the same internally in NiFi. The following example demonstrates normalizing
# DNs from certificates and principals from Kerberos into a common identity string:
#
# nifi.registry.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$
# nifi.registry.security.identity.mapping.value.dn=$1@$2
# nifi.registry.security.identity.mapping.transform.dn=NONE

# nifi.registry.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$
# nifi.registry.security.identity.mapping.value.kerb=$1@$2
# nifi.registry.security.identity.mapping.transform.kerb=UPPER

# Group Mapping Properties #
# These properties allow normalizing group names coming from external sources like LDAP. The following example
# lowercases any group name.
#
# nifi.registry.security.group.mapping.pattern.anygroup=^(.*)$
# nifi.registry.security.group.mapping.value.anygroup=$1
# nifi.registry.security.group.mapping.transform.anygroup=LOWER


# kerberos properties #
nifi.registry.kerberos.krb5.file=
nifi.registry.kerberos.spnego.principal=
nifi.registry.kerberos.spnego.keytab.location=
nifi.registry.kerberos.spnego.authentication.expiration=12 hours

# OIDC #
nifi.registry.security.user.oidc.discovery.url=
nifi.registry.security.user.oidc.connect.timeout=
nifi.registry.security.user.oidc.read.timeout=
nifi.registry.security.user.oidc.client.id=
nifi.registry.security.user.oidc.client.secret=
nifi.registry.security.user.oidc.preferred.jwsalgorithm=

# revision management #
# This feature should remain disabled until a future NiFi release that supports the revision API changes
nifi.registry.revisions.enabled=false

[#8 · Accepted solution]

@lynott 

I see some issues in your configurations. And have some questions in order to address some other potential issues.

Authorizers.xml:

• File-user-group-provider

<identifier>file-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>
        <property name="Users File">./conf/users.xml</property>
        <property name="Initial User Identity 1">employee</property>
    </userGroupProvider>​

Here you are defining manually a "user identity" of "employee".    Is this "employee" a user that is authenticating into your NiFi via ldap authentication.  If so, I'd expect this user identity to be returned by the ldap-user-group-provider instead of the file-user-group-provider.  You can NOT have two user-group-provider return the same user identity.  So if "employee" is going to be returned by ldap-user-group-provider, it needs to be removed from file-user-group-provider.

NOTE: The file-user-group-provider will only generate the users.xml if it does not already exist.  So modifications to the configuration in the file-user-group-provider will NOT result in any changes to an already existing users.xml.  the users.xml file will only contain manually added user and group identities. user and group identities returned by other user-group-provider will not get added to the users.xml.

• ldap-user-group-provider
  I assume you ldap connection is good, so I'll just focus on the user sync and group sync areas:

        <property name="User Search Base"></property>
        <property name="User Object Class"></property>
        <property name="User Search Scope">SUBTREE</property>
        <property name="User Search Filter"></property>
        <property name="User Identity Attribute">cn</property>
        <property name="User Group Name Attribute"></property>
        <property name="User Group Name Attribute - Referenced Group Attribute"></property>

        <property name="Group Search Base">OU=COMPANYGroups,DC=corp,DC=company,DC=com</property>
        <property name="Group Object Class">group</property>
        <property name="Group Search Scope">ONE_LEVEL</property>
        <property name="Group Search Filter">((cn=nifi-admins))</property>
        <property name="Group Name Attribute">cn</property>
        <property name="Group Member Attribute">member</property>
        <property name="Group Member Attribute - Referenced User Attribute"></property>​

As configured you are performing a group sync that will return all the "member" attribute values from the "nifi-admins" group.  
1. Are the member attribute values full user DNs?  
Because you have "cn" configured in the user sync section "user Identity Attribute". each returned "member" is going to be looked up in order to get the cn value.
Since NiFi and NiFi-registry user and group identities are case sensitive, it is important that the user identities authorized match exactly with the user identity returned by user authentication in to NiFi-Registry.  I see from the nifi-registry.properties file that you are using the ldap-provider for user authentication.  Assuming you a re logging in using the user's "cn" as the username, you need to make sure that "USE_USERNAME" is being used and not "USE_DN" in that ldap-provider.

• Composite-provider:

    <userGroupProvider>
        <identifier>composite-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.CompositeUserGroupProvider</class>
        <property name="Configurable User Group Provider">file-user-group-provider</property>
        <property name="User Group Provider 1">ldap-user-group-provider</property>
    </userGroupProvider>

Here you are using the "composite-user-group-provider", but are trying to use a configurable provider with it.  Instead you should be using:

 <userGroupProvider>
        <identifier>composite-configurable-user-group-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider</class>
        <property name="Configurable User Group Provider">file-user-group-provider</property>
        <property name="User Group Provider 1">ldap-user-group-provider</property>
    </userGroupProvider>

 You'll notice the "class" is different between these two providers.  The "identifier" is not important, but must match the identifier set in your file-access-policy-provider.

• file-access-policy-prpovider

<identifier>file-access-policy-provider</identifier>
        <class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>
        <property name="User Group Provider">composite-user-group-provider</property>
        <property name="Authorizations File">./conf/authorizations.xml</property>
        <property name="Initial Admin Identity">employee</property>
        <property name="NiFi Group Name"></property>

        <!--<property name="NiFi Identity 1"></property>-->
    </accessPolicyProvider​

The file-access-policy provider uses the authorizations.xml file to store all configured authorizations.  It will only generate a authorizations.xml file if one does not already exist.  It can be configured with an "initial Admin Identity".  This configured Initial Admin Identity must be returned by one of yoru user-group-providers.  In your case "employee" is being used here.  The "NiFi Identity 1", "NiFi Identity 2", "NiFi Identity <xyz>" can be added to define your NiFi node certificate Identities which would allow this provider to seed the proxy policies needed by the NiFi hosts.  But keep in mind that the file-user-group-provider would also need to generate these identities for this to work. 

-------------------

users.xml and authorizations.xml files:
Your users.xml only contains one entry for "employee" which corresponds with your authorizers configuration.  That user is assigned a UUID.  I can then see that UUID properly authorized as and admin user in the authorizations.xml file.

What I do not see in the users.xml file is any group defined with uuid:

8c295cae-a773-4d6a-98cd-eef47d0b8189

2. I assume this is the uuid for your ldap-user-group-provider returned "nifi-admins" group?   
3. I assume you manually added this group to the authorizations.xml file since you mentioned you have been unable to manually configure user and policies from within the NiFi-Registry UI yet?

----------------------
NiFi-Registry UI:

4. When you login to the NiFi-Registry using user "employee" do you not see "employee" displayed in upper right corner of UI  along with the wrench icon?

Remember that only case sensitive "employee" and what ever group happens to be "8c295cae-a773-4d6a-98cd-eef47d0b8189" are currently authorized as an admins.

 5. I assume you obtained "8c295cae-a773-4d6a-98cd-eef47d0b8189" by enabling debug in the logback.xml for the ldap-user-group-provider class?  Or is this the uuid for the NiFi node certificate derived user identity?

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#9]

@MattWho 

  Thank you for your quick response, I have read over the above and will do my best to answer your questions, as the initial set up was done prior to me taking on the project. I have Updated the Authorizer.xml and removed 'employee' as the initial admin from the file-user-group-provider

In the identity-providers.xml <property name="Identity Strategy">USE_USERNAME</property> was already set correctly

I changed the class in the authorizers.xml for the composite-user-group-provider identity to org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider and made sure the file-access-policy-provider used the correct User group provider

the UUID in the authorizations.xml were already there when I began taking over the work on this project and I am going to assume they were added when the registry service was first started.
When accessing the Registry UI I do indeed see my user in the upper right hand corner.
Following these steps provided members of the Nifi admin group to be able to add a user in the UI.
I think where I may be going wrong here is the Nifi node certificate Identities. I added <property name="Nifi identity 1">CN=dev-nifi.example.com</property> to both the file-user-group-provider and the file-access-policy-provider. I think this may be an issue with how I am setting the Identity here perhaps, I also tried in the UI. I do now see three UUIDs in the Authorization.xml/users.xml after restarting the Nifi-registry.service on the EC2 instance; one for the 'employee' one for the group and one I am assuming is for the Nifi identity. 

[#10]

@lynott @sydney- 

Are you good now?  Your last response contains no questions.
If my previous response(s) helped with your initial issue, please a take a moment to click accepted on those responses.

Thank you,
Matt