Created on 05-17-2017 10:25 AM - edited 08-18-2019 02:47 AM
Hi everyone ,
Right now i'm trying to run my Nifi cluster using LDAP for many users. For each of my nodes have certificat and i have another one for my browser for admin stuff , which i have created with nfi-toolkit. Actually I'm able to connect to nifi with my admin account(I can add users, policies..).The problem accurs when a users tries to connect via LDAP. He gets this error:
this are the log :
2017-05-16 16:48:13,550 ERROR [NiFi Web Server-126] o.a.n.w.a.c.AdministrationExceptionMapper org.apache.nifi.admin.service.AdministrationException: Unable to validate the supplied credentials. Please contact the system administrator.. Returning Internal Server Error response. org.apache.nifi.admin.service.AdministrationException: Unable to validate the supplied credentials. Please contact the system administrator. at org.apache.nifi.web.api.AccessResource.createAccessToken(AccessResource.java:445) ~[classes/:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_112] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_112] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_112] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_112] at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) ~[jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) ~[jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) ~[jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) [jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) [jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) [jersey-server-1.19.jar:1.19] at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) [jersey-server-1.19.jar:1.19] at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) [jersey-servlet-1.19.jar:1.19] at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) [jersey-servlet-1.19.jar:1.19] at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) [jersey-servlet-1.19.jar:1.19] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [javax.servlet-api-3.1.0.jar:3.1.0] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1689) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517] at org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66) [classes/:na] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517] at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:207) [spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE] at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176) [spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE] at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) [spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE] at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) [spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517] at org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:51) [classes/:na] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [jetty-security-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1174) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1106) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:459) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.Server.handle(Server.java:524) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112] Caused by: org.apache.nifi.authentication.exception.IdentityAccessException: Unable to validate the supplied credentials. Please contact the system administrator. at org.apache.nifi.ldap.LdapProvider.authenticate(LdapProvider.java:340) ~[na:na] at org.apache.nifi.web.security.spring.LoginIdentityProviderFactoryBean$1.authenticate(LoginIdentityProviderFactoryBean.java:301) ~[nifi-web-security-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.web.api.AccessResource.createAccessToken(AccessResource.java:437) ~[classes/:na] ... 58 common frames omitted Caused by: org.springframework.security.authentication.InternalAuthenticationServiceException: hostname:636; nested exception is javax.naming.CommunicationException: hostame:636 [Root exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target] at org.springframework.security.ldap.authentication.LdapAuthenticationProvider.doAuthentication(LdapAuthenticationProvider.java:208) ~[na:na] at org.springframework.security.ldap.authentication.AbstractLdapAuthenticationProvider.authenticate(AbstractLdapAuthenticationProvider.java:82) ~[na:na] at org.apache.nifi.ldap.LdapProvider.authenticate(LdapProvider.java:310) ~[na:na] ... 60 common frames omitted Caused by: org.springframework.ldap.CommunicationException: hostname:636; nested exception is javax.naming.CommunicationException: hostname:636 [Root exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target] at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:108) ~[na:na] at org.springframework.ldap.core.support.AbstractContextSource.createContext(AbstractContextSource.java:356) ~[na:na] at org.springframework.ldap.core.support.AbstractContextSource.doGetContext(AbstractContextSource.java:140) ~[na:na] at org.springframework.ldap.core.support.AbstractContextSource.getReadOnlyContext(AbstractContextSource.java:159) ~[na:na] at org.springframework.ldap.core.LdapTemplate.executeReadOnly(LdapTemplate.java:802) ~[na:na] at org.springframework.security.ldap.SpringSecurityLdapTemplate.searchForSingleEntry(SpringSecurityLdapTemplate.java:316) ~[na:na] at org.springframework.security.ldap.search.FilterBasedLdapUserSearch.searchForUser(FilterBasedLdapUserSearch.java:126) ~[na:na] at org.springframework.security.ldap.authentication.BindAuthenticator.authenticate(BindAuthenticator.java:94) ~[na:na] at org.springframework.security.ldap.authentication.LdapAuthenticationProvider.doAuthentication(LdapAuthenticationProvider.java:189) ~[na:na] ... 62 common frames omitted Caused by: javax.naming.CommunicationException: hostname:636 at com.sun.jndi.ldap.Connection.<init>(Connection.java:226) ~[na:1.8.0_112] at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:137) ~[na:1.8.0_112] at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1614) ~[na:1.8.0_112] at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2746) ~[na:1.8.0_112] at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:319) ~[na:1.8.0_112] at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:192) ~[na:1.8.0_112] at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:210) ~[na:1.8.0_112] at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:153) ~[na:1.8.0_112] at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:83) ~[na:1.8.0_112] at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684) ~[na:1.8.0_112] at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:313) ~[na:1.8.0_112] at javax.naming.InitialContext.init(InitialContext.java:244) ~[na:1.8.0_112] at javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:154) ~[na:1.8.0_112] at org.springframework.ldap.core.support.LdapContextSource.getDirContextInstance(LdapContextSource.java:42) ~[na:na] at org.springframework.ldap.core.support.AbstractContextSource.createContext(AbstractContextSource.java:344) ~[na:na] ... 69 common frames omitted Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[na:1.8.0_112] at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949) ~[na:1.8.0_112] at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302) ~[na:1.8.0_112] at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296) ~[na:1.8.0_112] at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509) ~[na:1.8.0_112] at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) ~[na:1.8.0_112] at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) ~[na:1.8.0_112] at sun.security.ssl.Handshaker.process_record(Handshaker.java:914) ~[na:1.8.0_112] at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062) ~[na:1.8.0_112] at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) ~[na:1.8.0_112] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) ~[na:1.8.0_112] at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) ~[na:1.8.0_112] at com.sun.jndi.ldap.Connection.createSocket(Connection.java:376) ~[na:1.8.0_112] at com.sun.jndi.ldap.Connection.<init>(Connection.java:203) ~[na:1.8.0_112] ... 83 common frames omitted Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387) ~[na:1.8.0_112] at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) ~[na:1.8.0_112] at sun.security.validator.Validator.validate(Validator.java:260) ~[na:1.8.0_112] at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) ~[na:1.8.0_112] at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229) ~[na:1.8.0_112] at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124) ~[na:1.8.0_112] at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491) ~[na:1.8.0_112] ... 92 common frames omitted Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) ~[na:1.8.0_112] at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) ~[na:1.8.0_112] at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) ~[na:1.8.0_112] at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382) ~[na:1.8.0_112] ... 98 common frames omitted
And my login-identity-providers.xml
<loginIdentityProviders> <provider> <identifier>ldap-provider</identifier> <class>org.apache.nifi.ldap.LdapProvider</class> <property name="Authentication Strategy">START_TLS</property> <property name="Manager DN">cn=bindnifi,ou=Services,dc=mycompany</property> <property name="Manager Password">MOT DE PASSE</property> <property name="TLS - Keystore">/HDF/nifi-1.1.0.2.1.2.0-10/conf/certificat/keystore.jks</property> <property name="TLS - Keystore Password">MOT DE PASSE</property> <property name="TLS - Keystore Type">JKS</property> <property name="TLS - Truststore">/HDF/nifi-1.1.0.2.1.2.0-10/conf/certificat/truststore.jks</property> <property name="TLS - Truststore Password">MOT DE PASSE</property> <property name="TLS - Truststore Type">JKS</property> <property name="TLS - Client Auth">NONE</property> <property name="TLS - Protocol">TLS</property> <property name="TLS - Shutdown Gracefully"></property> <property name="Referral Strategy">IGNORE</property> <property name="Connect Timeout">10 secs</property> <property name="Read Timeout">10 secs</property> <property name="Url">ldap://hostname:636</property> <property name="User Search Base">dc=mycompany</property> <property name="User Search Filter">uid={0}</property> <property name="Identity Strategy">USE_USERNAME</property> <property name="Authentication Expiration">12 hours</property> </provider> </loginIdentityProviders>
I have already put the public key of my sercured LDAP server into my nifi keystore, i've tried many things that many post suggest and Each time I made changes and restarted NiFi, I deleted users.xml and authorizations.xml files before starting up NiFi again
Thanks
Toky
Created 05-18-2017 02:19 PM
We find out, what was the problem it was an issue about the LDAP certtificate we generate.
We changed it and now it work just fine anyway thanks for your time and your help 🙂
Toky
Created 05-17-2017 12:33 PM
I am really not sure what version of NiFi you are running. The Authentication strategy "LDAPS" is different then "START_TLS". LDAPS support in NiFi's login identifier is a fairly new addition. Consult your NiFi's imbedded admin guide documentation to verify that LDAPS is an option. If so you will want to first change your login-idnetity-providers.xml fiel configuration to use LDAPS instead of START_TLS. Next you will want to confirm your ldap servers URL. Typically ldaps urls start with ldaps:// instead of just ldap://.
You also mentioned that you add the ldaps servers public key to NiFi's keystore. NiFI use 2-way TLS authentication. You should have added the ldaps servers public key as a trustedCertEntry in NIFi's truststore instead of the keystore. Also you want to make sure you have added NiFi's nodes public key as a trustedCertEntry on the ldaps server as well. If all you NIfi certs where signed by a CA, you just need to add the public key for your CA as a trustedCertEntry instead.
Thank you,
Matt
Created 05-17-2017 05:14 PM
Thanks @Matt Clarke for your feedback.
We use nifi-1.1.0.2.1.2.0-10 and i think it supports Authentication strategy "LDAPS". In fact, about the url of LDAP, it was just a mistake in the post and not in the xml file. Since the CA is one of nifi's node, we added the public key of LDAP server to the trustedCertEntry of all Nifi nodes but not the other way. Do you think we really need to add all nifi nodes certificate into the LDAP server truststore ? When i'm tailing LDAP log i don't see anything and whatever we change in the login-provider the ERROR log doesn't change when we try to log in.
Thank
Toky
Created 05-17-2017 05:40 PM
The caused by line in the stack trace you provided points at a truststore issue most likely.
Causedby: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Try running the following command from the command line on the serevr running NiFi:
<JAVA_HOME>/bin/java -Djavax.net.ssl.trustStore=/<path to NiFi truststore>/truststore SSLPoke hostname:663
If it fails, use keytool to get a verbose output of both the keystore being used on your LDAPS server and Truststore being used by your NiFi.
keytool -v --list -keystore /<path>/<truststore or keystore>
Is the issuer of the PrivateKeyEntry in the LDAPS keystore found in NiFi's truststore as a trustedCertEntry?
Thanks,
Matt
Created 05-18-2017 09:09 AM
Hi @Matt Clarke
<JAVA_HOME>/bin/java -Djavax.net.ssl.trustStore=/<path to NiFi truststore>/truststore SSLPoke hostname:663
i get this error
Error: Could not find or load main class SSLPoke
In my LDAPS keystore there no sign of nifi public keys but in nifi truststore i can see the LDAPS server Key in nifi trustore
I' don't think it' relevant but in my server JAVA isn't installed, i just do export JAVA HOME= /path to Java8.
Thanks
Toky
Created 05-18-2017 02:19 PM
We find out, what was the problem it was an issue about the LDAP certtificate we generate.
We changed it and now it work just fine anyway thanks for your time and your help 🙂
Toky
Created 09-17-2023 02:54 AM
Looking at the resolution of your NiFi issues on LDAPS, I have the following questions around Keystore/truststore
1. I am currently using the same keystore/truststore that was internally generated by our identity management team when I had configured for singleuser mode for my LDAP configuration. Does a new one need to be generated for my initial admin user to be able to connect to NiFi? Right now I can't connect with initial admin user. My knowledge of the way keystore/truststore work is very scanty.
2. Can you share the present settings of your authorizers.xml, login-identity-provider.xml that is working (without passwords, of course). I am able to start NiFi with my settings but just can't connect with initial admin user. I have a feeling that it's keystore/truststore issue but can't seem to resolve it