Nifi Authorizers.xml -- Unable to locate user group provider with identifier composite-configurable-user-group-provider

[#1]

here is my configs , please let me know what is missing here 

<authorizers>

<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">/data05/nifi/conf/users.xml</property>
<property name="Legacy Authorized Users File" />
<property name="Initial User Identity 0">CN=Nifi_Admin,OU=NIFI</property>

</userGroupProvider>

<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">composite-configurable-user-group-provider</property>
<property name="Authorizations File">/data05/nifi/conf/authorizations.xml</property>
<property name="Initial Admin Identity">CN=Nifi_Admin,OU=NIFI</property>
<property name="Legacy Authorized Users File" />

<property name="Node Identity 1">CN=Nifi_Admin,OU=NIFI</property>
</accessPolicyProvider>

<authorizer>
<identifier>ranger-provider</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="Access Policy Provider">file-access-policy-provider</property>
</authorizer>
</authorizers>

error:

2020-01-16 06:43:15,531 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration': Unsatisfied dependency expressed through method 'setFilterChainProxySecurityConfigurer' parameter 1; nested exception is org.springframework.beans.factory.BeanExpressionException: Expression parsing failed; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.apache.nifi.web.NiFiWebApiSecurityConfiguration': Unsatisfied dependency expressed through method 'setJwtAuthenticationProvider' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jwtAuthenticationProvider' defined in class path resource [nifi-web-security-context.xml]: Cannot resolve reference to bean 'authorizer' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authorizer': FactoryBean threw exception on object creation; nested exception is org.apache.nifi.authorization.exception.AuthorizerCreationException: org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable to locate user group provider with identifier composite-configurable-user-group-provider
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:666)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:87)

[#2]

@pvillard @MattWho @TimothySpann @sunile_manjee  guys appreciate your inputs 

nifi is running in the background u=but UI not coming up, don't see any error in the log

I have enabled tls SSL certs on my three node cluster and enabled AD 

here are my configs looks like 

1) nifi.properties

cat nifi.properties
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Core Properties #
#nifi.flow.configuration.file=./conf/flow.xml.gz
nifi.flow.configuration.file=/data05/nifi/conf/flow.xml.gz
nifi.flow.configuration.archive.enabled=true
#nifi.flow.configuration.archive.dir=./conf/archive/
nifi.flow.configuration.archive.dir=/data05/nifi/conf/archive/
nifi.flow.configuration.archive.max.time=30 days
nifi.flow.configuration.archive.max.storage=500 MB
nifi.flow.configuration.archive.max.count=
nifi.flowcontroller.autoResumeState=true
nifi.flowcontroller.graceful.shutdown.period=10 sec
nifi.flowservice.writedelay.interval=500 ms
nifi.administrative.yield.duration=30 sec
# If a component has no work to do (is "bored"), how long should we wait before checking again for work?
nifi.bored.yield.duration=10 millis
nifi.queue.backpressure.count=10000
nifi.queue.backpressure.size=1 GB

#nifi.authorizer.configuration.file=./conf/authorizers.xml
#nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
#nifi.templates.directory=./conf/templates
#nifi.authorizer.configuration.file=/data05/nifi/conf/authorizers.xml
#nifi.login.identity.provider.configuration.file=/data05/nifi/conf/login-identity-providers.xml
#nifi.templates.directory=/data05/nifi/conf/templates

nifi.authorizer.configuration.file=/opt/nifi-1.9.2/conf/authorizers.xml
nifi.login.identity.provider.configuration.file=/opt/nifi-1.9.2/conf/login-identity-providers.xml
nifi.templates.directory=/opt/nifi-1.9.2/conf/templates

nifi.ui.banner.text=
nifi.ui.autorefresh.interval=30 sec
nifi.nar.library.directory=./lib
nifi.nar.library.autoload.directory=./extensions
nifi.nar.working.directory=./work/nar/
nifi.documentation.working.directory=./work/docs/components

####################
# State Management #
####################
#nifi.state.management.configuration.file=./conf/state-management.xml
nifi.state.management.configuration.file=/data05/nifi/conf/state-management.xml
# The ID of the local state provider
nifi.state.management.provider.local=local-provider
# The ID of the cluster-wide state provider. This will be ignored if NiFi is not clustered but must be populated if running in a cluster.
nifi.state.management.provider.cluster=zk-provider
# Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server
#nifi.state.management.embedded.zookeeper.start=false
nifi.state.management.embedded.zookeeper.start=true

# Properties file that provides the ZooKeeper properties to use if <nifi.state.management.embedded.zookeeper.start> is set to true
#nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties
nifi.state.management.embedded.zookeeper.properties=/data05/nifi/conf/zookeeper.properties

# H2 Settings
#nifi.database.directory=./database_repository
nifi.database.directory=/data04/nifi/database_repo/prd
nifi.h2.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE

# FlowFile Repository
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.wal.implementation=org.apache.nifi.wali.SequentialAccessWriteAheadLog
nifi.flowfile.repository.directory=/data02/nifi/flowfile_repo/prd
nifi.flowfile.repository.partitions=256
nifi.flowfile.repository.checkpoint.interval=2 mins
nifi.flowfile.repository.always.sync=false

nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000
nifi.swap.in.period=5 sec
nifi.swap.in.threads=1
nifi.swap.out.period=5 sec
nifi.swap.out.threads=4

# Content Repository
nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository
nifi.content.claim.max.appendable.size=1 MB
nifi.content.claim.max.flow.files=100
nifi.content.repository.directory.default=/data03/nifi/cont_repo/prd
nifi.content.repository.archive.max.retention.period=12 hours
nifi.content.repository.archive.max.usage.percentage=50%
nifi.content.repository.archive.enabled=true
nifi.content.repository.always.sync=false
nifi.content.viewer.url=../nifi-content-viewer/

# Provenance Repository Properties
nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
nifi.provenance.repository.debug.frequency=1_000_000
nifi.provenance.repository.encryption.key.provider.implementation=
nifi.provenance.repository.encryption.key.provider.location=
nifi.provenance.repository.encryption.key.id=
nifi.provenance.repository.encryption.key=

# Persistent Provenance Repository Properties
nifi.provenance.repository.directory.default=/data01/nifi/provenance_repo/prd
#nifi.provenance.repository.max.storage.time=24 hours
nifi.provenance.repository.max.storage.time=3 days
#nifi.provenance.repository.max.storage.size=1 GB
nifi.provenance.repository.max.storage.size=10 GB
#nifi.provenance.repository.rollover.time=30 secs
nifi.provenance.repository.rollover.time=1 min
#nifi.provenance.repository.rollover.size=100 MB
nifi.provenance.repository.rollover.size=512 MB
nifi.provenance.repository.query.threads=2
#nifi.provenance.repository.index.threads=2
nifi.provenance.repository.index.threads=4
nifi.provenance.repository.compress.on.rollover=true
nifi.provenance.repository.always.sync=false
# Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are:
# EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, Relationship, Details
nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship
# FlowFile Attributes that should be indexed and made searchable. Some examples to consider are filename, uuid, mime.type
nifi.provenance.repository.indexed.attributes=
# Large values for the shard size will result in more Java heap usage when searching the Provenance Repository
# but should provide better performance
#nifi.provenance.repository.index.shard.size=500 MB
nifi.provenance.repository.index.shard.size=2 GB
# Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from
# the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved.
nifi.provenance.repository.max.attribute.length=65536
nifi.provenance.repository.concurrent.merge.threads=2
nifi.provenance.repository.warm.cache.frequency=1 hour

# Volatile Provenance Respository Properties
nifi.provenance.repository.buffer.size=100000

# Component Status Repository
nifi.components.status.repository.implementation=org.apache.nifi.controller.status.history.VolatileComponentStatusRepository
nifi.components.status.repository.buffer.size=1440
nifi.components.status.snapshot.frequency=1 min

# Site to Site properties
nifi.remote.input.host=ip-172-31-38-192.us-west-2.compute.internal
#nifi.remote.input.secure=false
nifi.remote.input.secure=true
#nifi.remote.input.socket.port=
nifi.remote.input.socket.port=9997
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
#nifi.remote.input.http.transaction.ttl=60 secs
nifi.remote.contents.cache.expiration=30 secs

# web properties #
nifi.web.war.directory=./lib
#nifi.web.http.host=ip-172-31-38-192.us-west-2.compute.internal
#nifi.web.http.port=8080
nifi.web.http.network.interface.default=
#nifi.web.https.host=
nifi.web.https.host=ip-172-31-38-192.us-west-2.compute.internal
#nifi.web.https.port=
nifi.web.https.port=9696
nifi.web.https.network.interface.default=
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
nifi.web.max.header.size=16 KB
nifi.web.proxy.context.path=
nifi.web.proxy.host=

# security properties #
nifi.sensitive.props.key=
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
nifi.sensitive.props.provider=BC
nifi.sensitive.props.additional.keys=

##Through toolkit###

nifi.security.keystore=/opt/nifi-1.9.2/conf/keystore.jks
nifi.security.keystoreType=jks
nifi.security.keystorePasswd=asHvHZPz6wkciFP89a30Aseb/HBF/oFjvpqtORhfdtA
nifi.security.keyPasswd=asHvHZPz6wkciFP89a30Aseb/HBF/oFjvpqtORhfdtA
nifi.security.truststore=/opt/nifi-1.9.2/conf/truststore.jks
nifi.security.truststoreType=jks
nifi.security.truststorePasswd=r+QbYY3qj/LfAxYXsXpjjse5b5ey8ZAKL72815byLkI
nifi.security.needClientAuth=true
nifi.security.user.authorizer=file-provider
nifi.security.user.login.identity.provider=ldap-provider
nifi.security.ocsp.responder.url=
nifi.security.ocsp.responder.certificate=

# OpenId Connect SSO Properties #
nifi.security.user.oidc.discovery.url=
nifi.security.user.oidc.connect.timeout=5 secs
nifi.security.user.oidc.read.timeout=5 secs
nifi.security.user.oidc.client.id=
nifi.security.user.oidc.client.secret=
nifi.security.user.oidc.preferred.jwsalgorithm=

# Apache Knox SSO Properties #
nifi.security.user.knox.url=
nifi.security.user.knox.publicKey=
nifi.security.user.knox.cookieName=hadoop-jwt
nifi.security.user.knox.audiences=

# nifi.security.group.mapping.pattern.anygroup=^(.*)$
# nifi.security.group.mapping.value.anygroup=$1
# nifi.security.group.mapping.transform.anygroup=LOWER

# cluster common properties (all nodes must have same values) #
#nifi.cluster.protocol.heartbeat.interval=5 sec
nifi.cluster.protocol.heartbeat.interval=30 sec
#nifi.cluster.protocol.is.secure=false
nifi.cluster.protocol.is.secure=true

# cluster node properties (only configure for cluster nodes) #
#nifi.cluster.is.node=false
nifi.cluster.is.node=true
#nifi.cluster.node.address=
nifi.cluster.node.address=ip-xxx.us-west-2.compute.internal
nifi.cluster.node.protocol.port=9999
nifi.cluster.node.protocol.threads=30
nifi.cluster.node.protocol.max.threads=50
nifi.cluster.node.event.history.size=25
nifi.cluster.node.connection.timeout=50 sec
nifi.cluster.node.read.timeout=50 sec
nifi.cluster.node.max.concurrent.requests=100
nifi.cluster.firewall.file=
nifi.cluster.flow.election.max.wait.time=5 mins
nifi.cluster.flow.election.max.candidates=3

# cluster load balancing properties #
nifi.cluster.load.balance.host=
nifi.cluster.load.balance.port=6342
nifi.cluster.load.balance.connections.per.node=4
nifi.cluster.load.balance.max.thread.count=8
nifi.cluster.load.balance.comms.timeout=30 sec

# zookeeper properties, used for cluster management #
nifi.zookeeper.connect.string=ip-xxx.us-west-2.compute.internal:2181,ip-xx.us-west-2.compute.internal:2181,ip-1xx.us-west-2.compute.internal:2181
nifi.zookeeper.connect.timeout=30 secs
nifi.zookeeper.session.timeout=30 secs
nifi.zookeeper.root.node=/nifi

# Zookeeper properties for the authentication scheme used when creating acls on znodes used for cluster management
# Values supported for nifi.zookeeper.auth.type are "default", which will apply world/anyone rights on znodes
# and "sasl" which will give rights to the sasl/kerberos identity used to authenticate the nifi node
# The identity is determined using the value in nifi.kerberos.service.principal and the removeHostFromPrincipal
# and removeRealmFromPrincipal values (which should align with the kerberos.removeHostFromPrincipal and kerberos.removeRealmFromPrincipal
# values configured on the zookeeper server).
nifi.zookeeper.auth.type=
nifi.zookeeper.kerberos.removeHostFromPrincipal=
nifi.zookeeper.kerberos.removeRealmFromPrincipal=

#nifi.zookeeper.auth.type=sasl
#nifi.zookeeper.kerberos.removeHostFromPrincipal=true
#nifi.zookeeper.kerberos.removeRealmFromPrincipal=true

# kerberos #
nifi.kerberos.krb5.file=

# kerberos service principal #
nifi.kerberos.service.principal=
nifi.kerberos.service.keytab.location=

# kerberos spnego principal #
nifi.kerberos.spnego.principal=
nifi.kerberos.spnego.keytab.location=
nifi.kerberos.spnego.authentication.expiration=12 hours

# external properties files for variable registry
# supports a comma delimited list of file locations
nifi.variable.registry.properties=

2) login-identity-provider.xml

<loginIdentityProviders>

<provider>
<identifier>ldap-provider</identifier>
<class>org.apache.nifi.ldap.LdapProvider</class>
<property name="Identity Strategy">USE_USERNAME</property>
<property name="Authentication Strategy">SIMPLE</property>
<property name="Manager DN">xxxxxx</property>
<property name="Manager Password">xxx</property>
<property name="TLS - Keystore"/>
<property name="TLS - Keystore Password"/>
<property name="TLS - Keystore Type"/>
<property name="TLS - Truststore"/>
<property name="TLS - Truststore Password"/>
<property name="TLS - Truststore Type"/>
<property name="TLS - Client Auth"/>
<property name="TLS - Protocol"/>
<property name="TLS - Shutdown Gracefully"/>
<property name="Referral Strategy">FOLLOW</property>
<property name="Connect Timeout">10 secs</property>
<property name="Read Timeout">10 secs</property>
<property name="Url">ldap://xx.ad.xx.edu:xx</property>
<property name="User Search Base">dc=xx,dc=xxx,dc=xx</property>
<property name="User Search Filter">sAMAccountName={0}</property>
<property name="Identity Strategy">USE_USERNAME</property>
<property name="Authentication Expiration">12 hours</property>
</provider>

</loginIdentityProviders>

3) authorizers.xml

<authorizers>
<authorizer>
<identifier>file-provider</identifier>
<class>org.apache.nifi.authorization.FileAuthorizer</class>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Users File">./conf/users.xml</property>
<property name="Initial Admin Identity">CN=Nifi_Admin,OU=NIFI</property>
<property name="Legacy Authorized Users File"></property>

<property name="Node Identity 1">CN=Nifi_Admin,OU=NIFI</property>
<property name="Node Identity 2">CN=Nifi_Admin,OU=NIFI</property>
<property name="Node Identity 3">CN=Nifi_Admin,OU=NIFI</property>
</authorizer>
</authorizers>

[#3]

here is the log rotation and no errors but when i list for https port 9696 its not listening  

[root@ip-172-31-35-116 conf]# netstat | grep -i 9696

te.internal:9696, state=CONNECTED, updateId=11] to NodeConnectionStatus[nodeId=ip-172-31-35-116.us-west-2.compute.internal:9696, state=CONNECTED, updateId=11]
2020-01-16 07:57:49,836 INFO [Process Cluster Protocol Request-13] o.a.n.c.p.impl.SocketProtocolListener Finished processing request 798f4507-f2ab-47d3-bca9-3e64de45da5c (type=NODE_STATUS_CHANGE, length=1369 bytes) from ip-172-31-38-192.us-west-2.compute.internal in 97 millis
2020-01-16 07:57:49,837 INFO [Process Cluster Protocol Request-29] o.a.n.c.p.impl.SocketProtocolListener Finished processing request 6d541bc2-26ba-4c8b-ae3b-679a8846d62c (type=NODE_STATUS_CHANGE, length=1369 bytes) from ip-172-31-38-192.us-west-2.compute.internal in 96 millis
2020-01-16 07:57:49,898 INFO [Process Cluster Protocol Request-21] o.a.n.c.c.node.NodeClusterCoordinator Status of ip-172-31-38-192.us-west-2.compute.internal:9696 changed from NodeConnectionStatus[nodeId=ip-172-31-38-192.us-west-2.compute.internal:9696, state=CONNECTED, updateId=12] to NodeConnectionStatus[nodeId=ip-172-31-38-192.us-west-2.compute.internal:9696, state=CONNECTED, updateId=12]
2020-01-16 07:57:49,899 INFO [Process Cluster Protocol Request-15] o.a.n.c.c.node.NodeClusterCoordinator Status of ip-172-31-38-192.us-west-2.compute.internal:9696 changed from NodeConnectionStatus[nodeId=ip-172-31-38-192.us-west-2.compute.internal:9696, state=CONNECTED, updateId=12] to NodeConnectionStatus[nodeId=ip-172-31-38-192.us-west-2.compute.internal:9696, state=CONNECTED, updateId=12]
2020-01-16 07:57:49,900 INFO [Process Cluster Protocol Request-17] o.a.n.c.c.node.NodeClusterCoordinator Status of ip-172-31-45-114.us-west-2.compute.internal:9696 changed from NodeConnectionStatus[nodeId=ip-172-31-45-114.us-west-2.compute.internal:9696, state=CONNECTED, updateId=10] to NodeConnectionStatus[nodeId=ip-172-31-45-114.us-west-2.compute.internal:9696, state=CONNECTED, updateId=10]
2020-01-16 07:57:49,902 INFO [Process Cluster Protocol Request-15] o.a.n.c.p.impl.SocketProtocolListener Finished processing request c82e760c-04f9-46cf-aee2-2676215b7d37 (type=NODE_STATUS_CHANGE, length=1369 bytes) from ip-172-31-38-192.us-west-2.compute.internal in 126 millis
2020-01-16 07:57:49,902 INFO [Process Cluster Protocol Request-17] o.a.n.c.p.impl.SocketProtocolListener Finished processing request ccc9a53b-0a5d-4621-98c5-825c4dfabc69 (type=NODE_STATUS_CHANGE, length=1369 bytes) from ip-172-31-38-192.us-west-2.compute.internal in 117 millis
2020-01-16 07:57:49,902 INFO [Process Cluster Protocol Request-21] o.a.n.c.p.impl.SocketProtocolListener Finished processing request 23f5c858-a99b-434a-8a2d-63219e3ddf2c (type=NODE_STATUS_CHANGE, length=1369 bytes) from ip-172-31-38-192.us-west-2.compute.internal in 126 millis
2020-01-16 07:57:49,914 INFO [Process Cluster Protocol Request-14] o.a.n.c.c.node.NodeClusterCoordinator Status of ip-172-31-35-116.us-west-2.compute.internal:9696 changed from NodeConnectionStatus[nodeId=ip-172-31-35-116.us-west-2.compute.internal:9696, state=CONNECTED, updateId=11] to NodeConnectionStatus[nodeId=ip-172-31-35-116.us-west-2.compute.internal:9696, state=CONNECTED, updateId=11]
2020-01-16 07:57:49,925 INFO [Process Cluster Protocol Request-14] o.a.n.c.p.impl.SocketProtocolListener Finished processing request b828b7b5-1f44-4230-bd5a-d72b1f07f0da (type=NODE_STATUS_CHANGE, length=1369 bytes) from ip-172-31-38-192.us-west-2.compute.internal in 113 millis
2020-01-16 07:57:49,945 INFO [Process Cluster Protocol Request-6] o.a.n.c.c.node.NodeClusterCoordinator Status of ip-172-31-38-192.us-west-2.compute.internal:9696 changed from NodeConnectionStatus[nodeId=ip-172-31-38-192.us-west-2.compute.internal:9696, state=CONNECTED, updateId=12] to NodeConnectionStatus[nodeId=ip-172-31-38-192.us-west-2.compute.internal:9696, state=CONNECTED, updateId=12]
2020-01-16 07:57:49,952 INFO [Process Cluster Protocol Request-20] o.a.n.c.c.node.NodeClusterCoordinator Status of ip-172-31-45-114.us-west-2.compute.internal:9696 changed from NodeConnectionStatus[nodeId=ip-172-31-45-114.us-west-2.compute.internal:9696, state=CONNECTED, updateId=10] to NodeConnectionStatus[nodeId=ip-172-31-45-114.us-west-2.compute.internal:9696, state=CONNECTED, updateId=10]
^C
[root@ip-172-31-35-116 conf]# netstat | grep -i 9696

[#4]

@venkii 

Your first authorizers.xml file was not configured correctly which prevented your startup.
You "authorizer" (ranger-provider) was looking for the "file-access-provider" (which exists) and is configured to look for the "composite-configurable-user-group-provider" (which does not exist).
The file-access-provider should have been configured to look for the "file-user-group-provider" which you do have.  

As far as the Ui not be accessible once you switched to an entirely different authorizer...
The UI is the last thing that becomes available during the startup process.

Until you see the following lines output in the nifi-app.log, the application has not completed startup yet.

<timestamp> INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs:
<timestamp> INFO [main] org.apache.nifi.web.server.JettyServer <a href="https://ip-172-31-38-192.us-west-2.compute.internal:9696/nifi" target="_blank">https://ip-172-31-38-192.us-west-2.compute.internal:9696/nifi</a>

Hope this helps,

Matt

[#5]

HI @MattWho i have configured authorizers.xml by referring this document available by @pvillard 

could you check if the below updated conf file looks good?

authorizers.xml
<authorizers>

<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">/data05/nifi/conf/users.xml</property>
<property name="Legacy Authorized Users File" />
<property name="Initial User Identity 1">CN=Nifi_Admin,OU=NIFI</property>
<property name="Initial User Identity 2">CN=Nifi_Admin,OU=NIFI</property>
<property name="Initial User Identity 3">CN=Nifi_Admin,OU=NIFI</property>

</userGroupProvider>

<userGroupProvider>
<identifier>ldap-user-group-provider</identifier>
<class>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</class>
<property name="Authentication Strategy">SIMPLE</property>
<property name="Manager DN">CN=ap-unix-ldap,OU=UNIX Service Accounts,OU=Service Accounts,OU=Accounts,OU=New OU Structure,OU=AD Infrastructure,DC=AD,DC=XX,DC=XX</property>
<property name="Manager Password">>XXXXX</property>
<property name="Referral Strategy">FOLLOW</property>
<property name="Connect Timeout">10 secs</property>
<property name="Read Timeout">10 secs</property>
<property name="Page size"/>
<property name="Sync Interval">20 min</property>
<property name="Url">ldap://ldap.ad.XXX.edu:389</property>
<property name="User Search Base">DC=AD,DC=XXX,DC=XX</property>
<property name="User Object Class">user</property>
<property name="User Search Scope">SUBTREE</property>
<property name="User Search Filter">(|(memberOf=CN=ITS-UAP,OU=Official Groups,OU=ITS,DC=AD,DC=XXX,DC=XXX)(memberOf=CN=nifi-users,OU=CI-UBPS,OU=AD Infrastructure,DC=AD,DC=XXX,DC=XXX))</property>
<property name="User Identity Attribute">sAMAccountName</property>
<property name="User Group Name Attribute"/>
<property name="User Group Name Attribute - Referenced Group Attribute"/>

<property name="Group Search Base">DC=AD,DC=XX,DC=XX</property>
<property name="Group Object Class">group</property>
<property name="Group Search Scope">SUBTREE</property>
<property name="Group Search Filter">(|(CN=nifi-users)(CN=ITS-UAP))</property>
<property name="Group Name Attribute">cn</property>
<property name="Group Member Attribute">member</property>
<property name="Group Member Attribute - Referenced User Attribute"/>
</userGroupProvider>

<userGroupProvider>

<identifier>composite-configurable-user-group-provider</identifier>
<class>org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider</class>
<property name="Configurable User Group Provider">file-user-group-provider</property>
<property name="User Group Provider 1">ldap-user-group-provider</property>
</userGroupProvider>

<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">composite-configurable-user-group-provider</property>
<property name="Authorizations File">/data05/nifi/conf/authorizations.xml</property>
<property name="Initial Admin Identity">CN=admin, OU=NIFI</property>
<property name="Legacy Authorized Users File" />

<property name="Node Identity 1">CN=Nifi_Admin,OU=NIFI</property>
<property name="Node Identity 2">CN=Nifi_Admin,OU=NIFI</property>
<property name="Node Identity 3">CN=Nifi_Admin,OU=NIFI</property>
</accessPolicyProvider>

<authorizer>
<identifier>ranger-provider</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="Access Policy Provider">file-access-policy-provider</property>
</authorizer>
</authorizers>

[#6]

here is the error I'm facing now, looks like AD access issue or firewall issues

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jwtAuthenticationProvider' defined in class path resource [nifi-web-security-context.xml]: Cannot resolve reference to bean 'authorizer' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authorizer': FactoryBean threw exception on object creation; nested exception is org.springframework.ldap.CommunicationException: ldap.ad.xx.xx:389; nested exception is javax.naming.CommunicationException: ldap.ad.xx.xx:389 [Root exception is java.net.SocketTimeoutException: connect timed out]
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)

[#7]

@venkii 

During NiFi startup as NiFi loads the authorizers.xml it will use the configured "ldap-user-group-provider" to seed NiFi will a list of users and groups based upon that providers configuration.

The error is telling you that the connection attempted based on your configuration timed out to ldap.ad.xx.xx:389.   I would verify that every node in your NiFi cluster can resolve your ldap hostname and and there are no network firewalls blocking any of your nodes from connecting.

As far as your ldap-user-group-provider configuration goes...
1. Perhaps adjust your Connect and Read timeouts to see if that helps.
2. Your Sync interval is aggressive at every 20 minutes.  Are new users and groups commonly added to ldap that often?
3. Not really sure how many users are part of your two groups, but if it is a lot you will want to configure a page size (500 for example)

4.  You did not set your "User Group Name Attribute" which based on your other configuration should be "memberOf"

Additionally...
Why are all your Initial user identities and Node Identities set to "CN=Nifi_Admin,OU=NIFI" . These should be unique DNs for each of the nodes in your cluster.
Your initial Admin Identity in the file-access-policy-provider is set to CN=admin, OU=NIFI.  Since this initial Admin identity is also not configured in the file-user-group-provider, it is expected that this user is one of your ldap users.  Problem here is that your ldap-user-group-provider is using sAMAccountName value as the user string and that value will not contain a full DN like you have configured.

Anytime you start NiFi without existing users.xml or authorizations.xml files, they will be built by the authorizer.  If they already exist they will not get modified or replaced.  So if you are still editing the authorizers.xml file to get your setup working, you may need to delete these two files from each node before each restart attempt if you make any changes to the Initial User Identities, Node Identities, and/or Initial Admin Identity.  NOTE:  authorizers.xml and authorizations.xml files are two totally different files. Make sure you do not delete the wrong one.

Hope this info is helpful,

Matt

[#8]

i have generated certs through toolkit 

tls-toolkit.sh standalone -n 'localhost(3)' -C 'CN=Nifi_Admin,OU=NIFI' -O -o /home/ec2-user/

will this not work?

<property name="Initial User Identity 1">CN=Nifi_Admin,OU=NIFI</property>
<property name="Initial User Identity 2">CN=Nifi_Admin,OU=NIFI</property>
<property name="Initial User Identity 3">CN=Nifi_Admin,OU=NIFI</property>

<property name="Node Identity 1">CN=Nifi_Admin,OU=NIFI</property>
<property name="Node Identity 2">CN=Nifi_Admin,OU=NIFI</property>
<property name="Node Identity 3">CN=Nifi_Admin,OU=NIFI</property>

[#9]

@MattWho As per this document, have to generate certs for server and client @pvillard 

https://pierrevillard.com/2016/11/29/apache-nifi-1-1-0-secured-cluster-setup/

whereas this shows more granular 

one cert shared across all nodes in a cluster

https://community.cloudera.com/t5/Community-Articles/Using-the-TLS-Toolkit-to-simplify-security/ta-p...

please recommend 

[#10]

@venkii 

You need to generate a certificate for each of your NiFi nodes in your NiFi cluster.

While you can create just one certificate (single DN) with multiple Subject Alternative Names (SAN) entries for each host, this is not security best practices.

There is no point in setting multiple identities for the same DN string.

For each "Initial User Identity <num>", the configured string is added as a user in the users.xml file
for each "Node Identity <num>", the configured string has authorizations needed for NiFi nodes associated to that user's uuid in the authorizations.xml file.

In order to access the NiFi UI after securing NiFi, your user/client will need to authenticate with your NiFi.  By default the only configured user authentication method setup is via TLS certificates which requires you to to create a certificate for each user accessing NiFi.  This certificate would be loaded in to your browser and not NiFi itself (the public certificate for your user's private certificate would need to be added to the NiFi truststore in order for NiFi to trust the cert).  

NiFi can also be configured to support kerberos, ldap, openId connect, etc alternative methods for user authentication.
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication

Once a user is successfully authenticated, that user must be authorized before they can even see the NiFi UI.
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#multi-tenant-authorization

I understand you have already started down this path, but I recommend you carefully read above document links for more details.