question Re: Nifi loading all the AD groups, but it doesn't retrieve members in it in Support Questions

Nifi loading all the AD groups, but it doesn't retrieve members in it

sha257 — Wed, 04 Dec 2024 14:21:11 GMT

Hi ,

My Nifi loads all the AD groups from my LDAP server , but it doensn't load any member information of it .

As per the image , one can see that the Members list is empty.

How is it possible to get the users in the group too ?

I can add it manually , but I would like the ldap to be able to retrieve the users as per the group.

Re: Nifi loading all the AD groups, but it doesn't retrieve members in it

MattWho — Wed, 04 Dec 2024 14:53:31 GMT

@sha257

This is typically caused by a bad configuration in the ldap-user-group-provider in the authorizers.xml.

It would be difficult for me to provide and configuration specifics with out a sample output from your AD for a group and one of the group members.

However, if you can share your ldap-user-group-provider configuration (minus any sensitive values), I may be able to offer some suggestions.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Re: Nifi loading all the AD groups, but it doesn't retrieve members in it

sha257 — Wed, 04 Dec 2024 20:38:55 GMT

Hi @MattWho ,

Thank you for your reply . Here is the ldap-user-group-provider from my authorizers.xml

<userGroupProvider> <identifier>ldap-user-group-provider</identifier> <class>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</class> <property name="Authentication Strategy">LDAPS</property> <property name="Manager DN">CN=ABC123,OU=Service Users,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Manager Password">xxxx</property> <property name="TLS - Keystore">/opt/nifi/nifi-current/conf/my-nifi-nifi-0.my-nifi-nifi-headless.default.svc.cluster.local/keystore.jks</property> <property name="TLS - Keystore Password">xxxx</property> <property name="TLS - Keystore Type">jks</property> <property name="TLS - Truststore">/opt/nifi/nifi-current/conf/my-nifi-nifi-0.my-nifi-nifi-headless.default.svc.cluster.local/truststore.jks</property> <property name="TLS - Truststore Password">xxx</property> <property name="TLS - Truststore Type">JKS</property> <property name="TLS - Client Auth">NONE</property> <property name="TLS - Protocol">TLS</property> <property name="TLS - Shutdown Gracefully">false</property> <property name="Referral Strategy">FOLLOW</property> <property name="Connect Timeout">10 secs</property> <property name="Read Timeout">10 secs</property> <property name="Url">ldaps://someldap.corp1.ad1.xyz.net:636</property> <property name="Page Size"></property> <property name="Sync Interval">2 mins</property> <property name="User Search Base">OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="User Object Class">person</property> <property name="User Search Scope">SUBTREE</property> <property name="User Search Filter">(memberOf=CN=AG-X-SAMPLE-ADMIN,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net)</property> <property name="User Identity Attribute">cn</property> <property name="User Group Name Attribute"></property> <property name="User Group Name Attribute - Referenced Group Attribute"></property> <property name="Group Search Base">OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Group Object Class">group</property> <property name="Group Search Scope">ONE_LEVEL</property> <property name="Group Search Filter">(|(cn=AG*)(cn=UG*))</property> <property name="Group Name Attribute"></property> <property name="Group Member Attribute"></property> <property name="Group Member Attribute - Referenced User Attribute">distinguishedName</property> </userGroupProvider>

and here is the sample from my ldapsearch :

21:27 $ ldapsearch -x -H ldaps://someldap.corp1.ad1.xyz.net:636 -D "CN=ABC123,OU=Service Users,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net" -w "xxxxx" -b "OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net" "(memberOf=CN=AG-X-SAMPLE-ADMIN,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net)" # extended LDIF # # LDAPv3 # base <OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net> with scope subtree # filter: (memberOf=CN=AG-X-SAMPLE-ADMIN,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net) # requesting: ALL # # W0YZ1, Solid Users, User Accounts, corp1.ad1.xyz.net dn: CN=W0YZ1,OU=Solid Users,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net objectClass: top objectClass: person objectClass: organizationalPerson objectClass: user cn: W0YZ1 sn: Jones c: XY l: Earth title: Developer, Platform givenName: Jack distinguishedName: CN=W0YZ1,OU=Solid Users,OU=User Accounts,DC=corp1,DC=ad1,D C=xyz,DC=net instanceType: 4 displayName: Jones, Jack uSNCreated: 123456 memberOf: CN=YG-COI-EMPL-PRD,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net memberOf: CN=AG-X-123456-ADMIN,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net memberOf: CN=AG-X-789101-ADMIN,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net memberOf: CN=AG-X-111213-AGENT,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net memberOf: CN=AG-X-131516-AGENT,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net memberOf: CN=AG-X-171819-AGENT,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net sAMAccountName: W0YZ1 mail: Jack.Jones@xyz.se

Please let me know in case you need additional information from me.

Thanks in advance for all you help !

Re: Nifi loading all the AD groups, but it doesn't retrieve members in it

MattWho — Thu, 05 Dec 2024 15:11:22 GMT

@sha257

NiFi's ldap-user-group-provider does not support nested groups (NIFI-8035)

Based on what you have shared, your user "W0YZ1" is not a direct memberOf group:

AG-X-SAMPLE-ADMIN

The user is a member of a bunch of other groups and I am guessing that one of more of these groups are a member of the above group.

And since your user search filter is only going to return users that are a direct memberOf:

CN=AG-X-SAMPLE-ADMIN,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net

NiFi's ldap-user-group-provider is not going to return the sample user you shared above.

Something you may want to try here:
It is not required that both the user and group search properties are configured in order to get users and groups returned. In your case I would suggest only performing the group sync to see what you get.
The following properties should be adjusted:

<property name="Page Size">500</property> <property name="Sync Interval">30 mins</property> <property name="User Search Base"></property> <property name="User Object Class"></property> <property name="User Search Scope">SUBTREE</property> <property name="User Search Filter"></property> <property name="User Identity Attribute">sAMAccountName</property> <property name="User Group Name Attribute"></property> <property name="User Group Name Attribute - Referenced Group Attribute"></property> <property name="Group Search Base">OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Group Object Class">group</property> <property name="Group Search Scope">SUBTREE</property> <property name="Group Search Filter">(|(cn=AG*)(cn=UG*))</property> <property name="Group Name Attribute">sAMAccountName</property> <property name="Group Member Attribute">member</property> <property name="Group Member Attribute - Referenced User Attribute"></property>

The above changes assume that your groups have "member" attribute.
I recommend always setting a page size of either 500 to avoid missing returns for large queries.
I do not recommend re-syncing users and groups every 2 mins as you had originally set. This adds unnecessary load on CPU. Keep in mind that all user and group identities synced are loaded into NiFi's heap memory.

The above setup will return all groups starting from the group search base that start with yoru configured search filter. From each returned group all the member lines will be returned which should contain the full DN for user members. Those returned DNS are then looked up to return the "sAMAccountName" string for each user member. I saw you were using full DNs for yoru groups previously (which is ok). The sAMAccountName string for your groups will also be used as the group identity.

You can also out the ldap-user-group-provider class in debug in the NiFi logback.xml to get output of the user and group identity strings synced with ease sync execution:

org.apache.nifi.ldap.tenants.LdapUserGroupProvider

Thank you,
Matt

Re: Nifi loading all the AD groups, but it doesn't retrieve members in it

sha257 — Fri, 06 Dec 2024 10:39:04 GMT

Hello @MattWho ,

Thank you for your response .

I have made the changes as you suggested , however I see the below Debug message each time :

2024-12-06 10:07:02,187 DEBUG [main] o.a.n.ldap.tenants.LdapUserGroupProvider Group member attribute [member] does not exist for [AG-RANDOMADMIN]. This may be due to misconfiguration or the group may not have any members. Ignoring group membership.

I do see that this group does not have any memebers at all .

Is there a way I can filter out or ignore the groups that do not have any members at all ?

The only other change I did was to change

<property name="Group Search Filter">(|(member=*)(uniqueMember=*))</property>

to do this filtering out , but Nifi service keeps dying after throwing some of these exceptions :

app-log 2024-12-06 10:28:47,557 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loading Application Properties [/opt/nifi/nifi-current/./conf/nifi.properties] app-log at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:214) app-log at org.springframework.aop.framework.autoproxy.BeanFactoryAdvisorRetrievalHelper.findAdvisorBeans(BeanFactoryAdvisorRetrievalHelper.java:91) app-log at org.springframework.aop.framework.autoproxy.AbstractAdvisorAutoProxyCreator.findCandidateAdvisors(AbstractAdvisorAutoProxyCreator.java:111) app-log at org.springframework.aop.aspectj.annotation.AnnotationAwareAspectJAutoProxyCreator.findCandidateAdvisors(AnnotationAwareAspectJAutoProxyCreator.java:92) app-log at org.springframework.aop.aspectj.autoproxy.AspectJAwareAdvisorAutoProxyCreator.shouldSkip(AspectJAwareAdvisorAutoProxyCreator.java:101) app-log at org.springframework.aop.framework.autoproxy.AbstractAutoProxyCreator.postProcessBeforeInstantiation(AbstractAutoProxyCreator.java:255) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInstantiation(AbstractAutowireCapableBeanFactory.java:1160) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.resolveBeforeInstantiation(AbstractAutowireCapableBeanFactory.java:1135) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:531) app-log ... 48 common frames omitted app-log Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration': Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authorizer': FactoryBean threw exception on object creation; nested exception is org.springframework.ldap.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 app-log ]; nested exception is javax.naming.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 app-log ]; remaining name 'CN=XYZ1234,OU=DummyAdministrators,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net' app-log at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:794) app-log at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:220) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) app-log at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:336) app-log at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) app-log at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:334) app-log at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:209) app-log at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:401) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getSingletonFactoryBeanForTypeCheck(AbstractAutowireCapableBeanFactory.java:1027) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getTypeForFactoryBean(AbstractAutowireCapableBeanFactory.java:907) app-log at org.springframework.beans.factory.support.AbstractBeanFactory.isTypeMatch(AbstractBeanFactory.java:638) app-log at org.springframework.beans.factory.support.DefaultListableBeanFactory.doGetBeanNamesForType(DefaultListableBeanFactory.java:583) app-log at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:550) app-log at org.springframework.beans.factory.BeanFactoryUtils.beanNamesForTypeIncludingAncestors(BeanFactoryUtils.java:265) app-log at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1557) app-log at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1354) app-log at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) app-log at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:904) app-log at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:781) app-log ... 74 common frames omitted app-log Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authorizer': FactoryBean threw exception on object creation; nested exception is org.springframework.ldap.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 app-log ]; nested exception is javax.naming.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 app-log ]; remaining name 'CN=XYZ1234,OU=DummyAdministrators,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net' app-log at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:176) app-log at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:101) app-log at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1899) app-log at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getObjectForBeanInstance(AbstractAutowireCapableBeanFactory.java:1284) app-log at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:346) app-log at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:209) app-log at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) app-log at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) app-log at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) app-log at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:904) app-log at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:781) app-log ... 97 common frames omitted app-log Caused by: org.springframework.ldap.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 app-log ]; nested exception is javax.naming.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 app-log ]; remaining name 'CN=XYZ1234,OU=DummyAdministrators,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net' app-log at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:212) app-log at org.springframework.ldap.core.LdapTemplate.executeWithContext(LdapTemplate.java:824) app-log at org.springframework.ldap.core.LdapTemplate.executeReadOnly(LdapTemplate.java:807) app-log at org.springframework.ldap.core.LdapTemplate.lookup(LdapTemplate.java:848) app-log at org.apache.nifi.ldap.tenants.LdapUserGroupProvider$3.doMapFromContext(LdapUserGroupProvider.java:620) app-log at org.apache.nifi.ldap.tenants.LdapUserGroupProvider$3.doMapFromContext(LdapUserGroupProvider.java:570) app-log at org.springframework.ldap.core.support.AbstractContextMapper.mapFromContext(AbstractContextMapper.java:43) app-log at org.springframework.ldap.core.ContextMapperCallbackHandler.getObjectFromNameClassPair(ContextMapperCallbackHandler.java:69) app-log at org.springframework.ldap.core.CollectingNameClassPairCallbackHandler.handleNameClassPair(CollectingNameClassPairCallbackHandler.java:50) app-log at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:371) app-log at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:332) app-log at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:633) app-log at org.apache.nifi.ldap.tenants.LdapUserGroupProvider.load(LdapUserGroupProvider.java:570) app-log at org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:386) app-log at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) app-log at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) app-log at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) app-log at java.base/java.lang.reflect.Method.invoke(Unknown Source) app-log at org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38) app-log at com.sun.proxy.$Proxy81.onConfigured(Unknown Source) app-log at org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:199) app-log at org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:168) app-log at org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:72) app-log at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169) app-log ... 107 common frames omitted app-log Caused by: javax.naming.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 app-log ] app-log at java.naming/com.sun.jndi.ldap.LdapCtx.mapErrorCode(Unknown Source) app-log at java.naming/com.sun.jndi.ldap.LdapCtx.processReturnCode(Unknown Source) app-log at java.naming/com.sun.jndi.ldap.LdapCtx.processReturnCode(Unknown Source) app-log at java.naming/com.sun.jndi.ldap.LdapCtx.c_lookup(Unknown Source) app-log at java.naming/com.sun.jndi.toolkit.ctx.ComponentContext.p_lookup(Unknown Source) app-log at java.naming/com.sun.jndi.toolkit.ctx.PartialCompositeContext.lookup(Unknown Source) app-log at java.naming/com.sun.jndi.toolkit.ctx.PartialCompositeContext.lookup(Unknown Source) app-log at java.naming/javax.naming.InitialContext.lookup(Unknown Source) app-log at java.base/jdk.internal.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) app-log at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) app-log at java.base/java.lang.reflect.Method.invoke(Unknown Source) app-log at org.springframework.ldap.core.support.SingleContextSource$NonClosingDirContextInvocationHandler.invoke(SingleContextSource.java:197) app-log at com.sun.proxy.$Proxy84.lookup(Unknown Source) app-log at org.springframework.ldap.core.LdapTemplate$10.executeWithContext(LdapTemplate.java:850) app-log at org.springframework.ldap.core.LdapTemplate.executeWithContext(LdapTemplate.java:821) app-log ... 129 common frames omitted app-log 2024-12-06 10:29:48,297 INFO [Thread-0] org.apache.nifi.NiFi Application Server shutdown started bootstrap-log 2024-12-06 10:29:48,296 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: Error creating bean with name 'niFiWebApiConfiguration': BeanPostProcessor before instantiation of bean failed; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.security.config.annotation.method.configuration.PrePostMethodSecurityConfiguration': Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.apache.nifi.web.security.configuration.AuthenticationSecurityConfiguration': Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authorizer': FactoryBean threw exception on object creation; nested exception is org.springframework.ldap.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 bootstrap-log 2024-12-06 10:29:48,296 ERROR [NiFi logging handler] org.apache.nifi.StdErr ]; nested exception is javax.naming.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 bootstrap-log 2024-12-06 10:29:48,296 ERROR [NiFi logging handler] org.apache.nifi.StdErr ]; remaining name 'CN=XYZ1234,OU=DummyAdministrators,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net' bootstrap-log 2024-12-06 10:29:48,296 ERROR [NiFi logging handler] org.apache.nifi.StdErr Shutting down... user-log 2024-12-06 10:29:48,077 ERROR [main] o.a.n.a.AuthorizerFactoryBean User Group Provider [composite-configurable-user-group-provider] destruction failed user-log java.lang.NullPointerException: null user-log at org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider.preDestruction(CompositeConfigurableUserGroupProvider.java:244) user-log at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) user-log at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) user-log at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) user-log at java.base/java.lang.reflect.Method.invoke(Unknown Source) user-log at org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38) user-log at com.sun.proxy.$Proxy80.preDestruction(Unknown Source) user-log at org.apache.nifi.authorization.AuthorizerFactoryBean.lambda$destroy$2(AuthorizerFactoryBean.java:566) user-log at java.base/java.util.HashMap.forEach(Unknown Source) user-log at org.apache.nifi.authorization.AuthorizerFactoryBean.destroy(AuthorizerFactoryBean.java:564) user-log at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:213) user-log at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:587) user-log at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:559) user-log at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1163) user-log at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:520) user-log at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1156) user-log at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1123) user-log at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:604) user-log at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:399) user-log at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:278) user-log at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103) user-log at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073) user-log at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) user-log at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002) user-log at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765) user-log at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) user-log at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) user-log at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) user-log at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916) user-log at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) user-log at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) user-log at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) user-log at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) user-log at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) user-log at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) user-log at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) user-log at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) user-log at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) user-log at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) user-log at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) user-log at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) user-log at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) user-log at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) user-log at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) user-log at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) user-log at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) user-log at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) user-log at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) user-log at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) user-log at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) user-log at org.eclipse.jetty.server.Server.start(Server.java:423) user-log at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) user-log at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) user-log at org.eclipse.jetty.server.Server.doStart(Server.java:387) user-log at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) user-log at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:818) user-log at org.apache.nifi.NiFi.<init>(NiFi.java:172) user-log at org.apache.nifi.NiFi.<init>(NiFi.java:83) user-log at org.apache.nifi.NiFi.main(NiFi.java:332) bootstrap-log 2024-12-06 10:29:49,275 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi

Re: Nifi loading all the AD groups, but it doesn't retrieve members in it

MattWho — Fri, 06 Dec 2024 21:11:20 GMT

@sha257
NiFi is failing to startup because the execution of the ldap-user-group-provider during startup failed to complete successfully.

Did you set the page size to 500 in the ldap-user-group-provider?

There really is no harm in syncing groups that have no members. They can't be used for authorization since a group can't authenticate into NiFi. All that matters is that the groups with members are getting synced and those members are listed for the group. Then group based authorizations can be established to control access for those member identities.

Keep in mind that the user and group identity strings being synced are loaded into NiFi heap memory. So you want to make sure your group search filter is syncing only the few groups containing user who will need to access your NiFi. So instead of using wildcards, declare the specific CNs for only the groups containing users that need to access your NiFi.

Typically NiFi specific groups are created in LDAP/AD for managing authorized access to NiFi.

Thank you,
Matt

Re: Nifi loading all the AD groups, but it doesn't retrieve members in it

sha257 — Mon, 09 Dec 2024 08:04:50 GMT

Hello @MattWho ,

Yes , I have set my page size as 500 . Please find my ldap-user-group-provider config below :

<userGroupProvider> <identifier>ldap-user-group-provider</identifier> <class>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</class> <property name="Authentication Strategy">LDAPS</property> <property name="Manager DN">CN=ABC123,OU=Service Users,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Manager Password">xxxx</property> <property name="TLS - Keystore">/opt/nifi/nifi-current/conf/my-nifi-nifi-0.my-nifi-nifi-headless.default.svc.cluster.local/keystore.jks</property> <property name="TLS - Keystore Password">xxxx</property> <property name="TLS - Keystore Type">jks</property> <property name="TLS - Truststore">/opt/nifi/nifi-current/conf/my-nifi-nifi-0.my-nifi-nifi-headless.default.svc.cluster.local/truststore.jks</property> <property name="TLS - Truststore Password">xxxx</property> <property name="TLS - Truststore Type">JKS</property> <property name="TLS - Client Auth">NONE</property> <property name="TLS - Protocol">TLS</property> <property name="TLS - Shutdown Gracefully">false</property> <property name="Referral Strategy">FOLLOW</property> <property name="Connect Timeout">10 secs</property> <property name="Read Timeout">10 secs</property> <property name="Url">ldaps://someldap.corp1.ad1.xyz.net:636</property> <property name="Page Size">500</property> <property name="Sync Interval">30 mins</property> <property name="User Search Base"></property> <property name="User Object Class">person</property> <property name="User Search Scope">SUBTREE</property> <property name="User Search Filter">(objectClass=*)</property> <property name="User Identity Attribute">sAMAccountName</property> <property name="User Group Name Attribute"></property> <property name="User Group Name Attribute - Referenced Group Attribute"></property> <property name="Group Search Base">OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Group Object Class">group</property> <property name="Group Search Scope">ONE_LEVEL</property> <property name="Group Search Filter">(|(member=*)(uniqueMember=*))</property> <property name="Group Name Attribute">sAMAccountName</property> <property name="Group Member Attribute">member</property> <property name="Group Member Attribute - Referenced User Attribute"></property> </userGroupProvider>

Leading me to the error

Caused by: org.springframework.ldap.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 │ │ app-log ]; nested exception is javax.naming.OperationNotSupportedException: [LDAP: error code 12 - 00002040: SvcErr: DSID-03140454, problem 5010 (UNAVAIL_EXTENSION), data 0 │ │ app-log ]; remaining name 'CN=xxxx,OU=LocalWorkstationAdministrators,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net' │ │ app-log at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:212) │ │ app-log at org.springframework.ldap.core.LdapTemplate.executeWithContext(LdapTemplate.java:824) │ │ app-log at org.springframework.ldap.core.LdapTemplate.executeReadOnly(LdapTemplate.java:807) │ │ app-log at org.springframework.ldap.core.LdapTemplate.lookup(LdapTemplate.java:848) │ │ app-log at org.apache.nifi.ldap.tenants.LdapUserGroupProvider$3.doMapFromContext(LdapUserGroupProvider.java:620) │ │ app-log at org.apache.nifi.ldap.tenants.LdapUserGroupProvider$3.doMapFromContext(LdapUserGroupProvider.java:570) │ │ app-log at org.springframework.ldap.core.support.AbstractContextMapper.mapFromContext(AbstractContextMapper.java:43) │ │ app-log at org.springframework.ldap.core.ContextMapperCallbackHandler.getObjectFromNameClassPair(ContextMapperCallbackHandler.java:69) │ │ app-log at org.springframework.ldap.core.CollectingNameClassPairCallbackHandler.handleNameClassPair(CollectingNameClassPairCallbackHandler.java:50) │ │ app-log at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:371) │ │ app-log at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:332) │ │ app-log at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:633) │ │ app-log at org.apache.nifi.ldap.tenants.LdapUserGroupProvider.load(LdapUserGroupProvider.java:570) │ │ app-log at org.apache.nifi.ldap.tenants.LdapUserGroupProvider.onConfigured(LdapUserGroupProvider.java:386) │ │ app-log at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) │ │ app-log at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) │ │ app-log at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) │ │ app-log at java.base/java.lang.reflect.Method.invoke(Unknown Source) │ │ app-log at org.apache.nifi.authorization.UserGroupProviderInvocationHandler.invoke(UserGroupProviderInvocationHandler.java:38) │ │ app-log at com.sun.proxy.$Proxy81.onConfigured(Unknown Source) │ │ app-log at org.apache.nifi.authorization.AuthorizerFactoryBean.loadProviderProperties(AuthorizerFactoryBean.java:199) │ │ app-log at org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:168) │ │ app-log at org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:72) │ │ app-log at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169) │ │ app-log ... 107 common frames omitted

Is there something else that I am missing here ?

Re: Nifi loading all the AD groups, but it doesn't retrieve members in it

MattWho — Mon, 09 Dec 2024 17:41:46 GMT

@sha257

You do NOT need to conduct both a user search and group search in order to get your users and groups. I suggested that you try only executing a group sync from which member users will be looked up. I see that you still have the a user search being executed.

Unset the following properties:

<property name="User Object Class">person</property> <property name="User Search Filter">(objectClass=*)</property>

leave only the following user sync properties set:

<property name="User Search Scope">SUBTREE</property> <property name="User Identity Attribute">sAMAccountName</property>

You can also add the following logger to the NiFi logback.xml file to see what user and group strings are being returned by this provider:

Without a group ldap entry example, I can only make suggestions.

Thank you,
Matt

Re: Nifi loading all the AD groups, but it doesn't retrieve members in it

sha257 — Mon, 09 Dec 2024 20:36:07 GMT

Thanks again @MattWho . I tried to incorporate the same , and the error still exists .

Additionally , I dont see much on the Debug logs , even after I have enabled it to get more info on that .

What kind of information could I provide you in regards to Ldap search and I can try to gather this information ?

Thanks

Re: Nifi loading all the AD groups, but it doesn't retrieve members in it

MattWho — Tue, 10 Dec 2024 14:09:27 GMT

@sha257

Did you restart your NiFi after modifying the logback.xml?

NiFi executes the ldap-user-group-provider during startup, so you should see DEBUG output in the nifi-app.log at that time and each time the sync interval happens.

As far as ldap sample, I would need to see a ldap search group output that contains "member" entries for users you expect to see.

Thanks,
Matt