question Re: SSL with Apache Nifi not working in Archives of Support Questions (Read Only)

SSL with Apache Nifi not working

Anishkumarv — Sat, 21 Jan 2017 00:44:46 GMT

I have been working on this problem for quite some time and I would like answers and suggestions from you guys on the issue I am facing. I am trying to get my Nifi standalone instance on my server and basically my requirement is LDAP authenitcation, for that i have read some documents and found need to setup SSL first so i have followed this link but i am getting the below error.

and my authorizers.xml

kindly have a look and advice.

	<authorizer>
        <identifier>file-provider</identifier>
        <class>org.apache.nifi.authorization.FileAuthorizer</class>
        <property name="Authorizations File">./conf/authorizations.xml</property>
        <property name="Users File">./conf/users.xml</property>
        <property name="Initial Admin Identity">"EMAILADDRESS=07anis@gmail.com,CN=admin,OU=EDM,O=XXX,L=HK,ST=HK,C=HK"</property>
        <property name="Legacy Authorized Users File"></property>
        <!-- Provide the identity (typically a DN) of each node when clustered, see above description of Node Identity.
        <property name="Node Identity 1"></property>
        <property name="Node Identity 2"></property>
        -->
    </authorizer>

2017-01-20 23:39:12,603 INFO [main] o.a.nifi.authorization.FileAuthorizer Authorizations file loaded at Fri Jan 20 23:39:12 HKT 2017
2017-01-20 23:39:59,327 INFO [NiFi Web Server-44] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response.
2017-01-20 23:39:59,331 DEBUG [NiFi Web Server-44] o.a.n.w.a.c.IllegalStateExceptionMapper
java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.
        at org.apache.nifi.web.api.AccessResource.createAccessTokenFromTicket(AccessResource.java:349) ~[classes/:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_77]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77]
        at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) [jersey-servlet-1.19.jar:1.19]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) [jersey-servlet-1.19.jar:1.19]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) [jersey-servlet-1.19.jar:1.19]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [javax.servlet-api-3.1.0.jar:3.1.0]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1689) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:51) [jetty-servlets-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66) [classes/:na]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:207) [spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176) [spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) [spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) [spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:51) [classes/:na]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [jetty-security-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1174) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1106) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.Server.handle(Server.java:524) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253) [jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [jetty-io-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) [jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) [jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) [jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) [jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) [jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
2017-01-20 23:39:59,556 DEBUG [NiFi Web Server-55] o.a.n.w.s.NiFiAuthenticationFilter Checking secure context token: null
2017-01-20 23:39:59,557 INFO [NiFi Web Server-55] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (EMAILADDRESS=07anis@gmail.com, CN=admin, OU=EDM, O=XXX, L=HK, ST=HK, C=HK) GET https://hklpadhod07:7443/nifi-api/flow/current-user (source ip: 10.112.194.113)
2017-01-20 23:39:59,559 INFO [NiFi Web Server-55] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for EMAILADDRESS=07anis@gmail.com, CN=admin, OU=EDM, O=XXX, L=HK, ST=HK, C=HK
2017-01-20 23:39:59,559 DEBUG [NiFi Web Server-55] o.a.n.w.s.NiFiAuthenticationFilter Checking secure context token: EMAILADDRESS=07anis@gmail.com, CN=admin, OU=EDM, O=XXX, L=HK, ST=HK, C=HK
2017-01-20 23:39:59,559 DEBUG [NiFi Web Server-55] o.a.n.w.s.NiFiAuthenticationFilter Checking secure context token: EMAILADDRESS=07anis@gmail.com, CN=admin, OU=EDM, O=XXX, L=HK, ST=HK, C=HK
2017-01-20 23:39:59,560 DEBUG [NiFi Web Server-55] o.a.n.w.s.a.NiFiAnonymousUserFilter SecurityContextHolder not populated with anonymous token, as it already contained: 'EMAILADDRESS=07anis@gmail.com, CN=admin, OU=EDM, O=XXX, L=HK, ST=HK, C=HK'
2017-01-20 23:39:59,571 INFO [NiFi Web Server-55] o.a.n.w.a.c.AccessDeniedExceptionMapper EMAILADDRESS=07anis@gmail.com, CN=admin, OU=EDM, O=XXX, L=HK, ST=HK, C=HK does not have permission to access the requested resource. Returning Forbidden response.
2017-01-20 23:39:59,572 DEBUG [NiFi Web Server-55] o.a.n.w.a.c.AccessDeniedExceptionMapper
org.apache.nifi.authorization.AccessDeniedException: Unknown user with identity 'EMAILADDRESS=07anis@gmail.com, CN=admin, OU=EDM, O=XXX, L=HK, ST=HK, C=HK'.
        at org.apache.nifi.web.api.FlowResource.authorizeFlow(FlowResource.java:226) ~[classes/:na]
        at org.apache.nifi.web.api.FlowResource.getCurrentUser(FlowResource.java:312) ~[classes/:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_77]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77]
        at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) [jersey-server-1.19.jar:1.19]
        at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) [jersey-servlet-1.19.jar:1.19]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) [jersey-servlet-1.19.jar:1.19]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) [jersey-servlet-1.19.jar:1.19]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [javax.servlet-api-3.1.0.jar:3.1.0]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1689) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:51) [jetty-servlets-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66) [classes/:na]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:316) [spring-security-web-4.0.3.RELEASE.jar:4.0s
<br>

Re: SSL with Apache Nifi not working

Anishkumarv — Sat, 21 Jan 2017 00:48:52 GMT

@Bryan Bende Can you help on this.

Re: SSL with Apache Nifi not working

bbende — Sat, 21 Jan 2017 00:48:59 GMT

Try the following...

Stop your NiFi
Delete users.xml and authorizations.xml
Edit authorizers.xml so that the "Initial Admin Identity" matches exactly what see in your logs, notice yours has no spaces and the logs do have spaces
Start again

Re: SSL with Apache Nifi not working

MattWho — Sat, 21 Jan 2017 00:57:49 GMT

@Anishkumar Valsalam

The log you provided shows that user authentication is successful. What is failing is the users authorization inside NiFi. When setting up NiFi securely for the first time (enable SSL), you need to provide an "Initial Admin Identity". The Identity provided here will be used to seed the admin related access policies (Authorizations) within NiFi which will give that user the ability to access the UI.

The issue you are running in to is that what your LDAP has returned and what you have configured in your authorizers.xml file do not match exactly.

Configured value: EMAILADDRESS=07anis@gmail.com,CN=admin,OU=EDM,O=XXX,L=HK,ST=HK,C=HK

LDAP value: EMAILADDRESS=07anis@gmail.com, CN=admin, OU=EDM, O=XXX, L=HK, ST=HK, C=HK

These are case sensitive and spaces do count.

You cannot simply update the authorizers.xml config file to resolve your issue. When NiFi is started securely for the first time, users.xml and authorizations.xml files are generated based on the configs in the authorizers.xml file. Subsequent restarts of NiFi will not trigger a modification or re-creation of these files if they already exist. So to correct your issue, you will need to correct your "Initial Admin Identity" configuration so it matches what is being returned by LDAP, remove the existing users.xml and authorizations.xml files and then restart so they get created with the correct value/user identity.

Once you have gained access to your NiFi UI, you can add additional users and set authorizations for those users directly form the UI.

Thanks,

Matt

Re: SSL with Apache Nifi not working

Anishkumarv — Sat, 21 Jan 2017 01:05:13 GMT

Thanks Matt and Brayan.

It resolved my current issue 🙂 thanks much. next step will connect to LDAP

can i get the any hortonworks knowledge link for ldap configuration with nifi.

and without kerberos/ldap is there any way to authenticate users example PAM.

Re: SSL with Apache Nifi not working

Anishkumarv — Sat, 21 Jan 2017 01:17:53 GMT

@Matt @Bryan Bende

Re: SSL with Apache Nifi not working

MattWho — Sat, 21 Jan 2017 01:23:32 GMT

@Anishkumar Valsalam

NiFi user authentication supports LDAP, Kerberos, or Users TLS certificates (default) only. I thought you were already using LDAP for user authentication? What do you have configured in your login-identity-providers.xml?

Matt

Re: SSL with Apache Nifi not working

Anishkumarv — Sat, 21 Jan 2017 01:50:13 GMT

Attached my login-identity-providers.xml FYR.

As of now i am in the beginnig step to create LDAP authentication i didnt enable anything.

Share any KL you have.

And

1) Once i authenticated with LDAP then for each ldap user do i need create ssl cert and to accept in browser to access NIFI?

Re: SSL with Apache Nifi not working

Anishkumarv — Sat, 21 Jan 2017 21:26:24 GMT

Hi Matt,

If i confiugre through LDAP at that time the TLS which i configured is required?

Re: SSL with Apache Nifi not working

Anishkumarv — Tue, 24 Jan 2017 06:59:08 GMT

@Bryan Bende

Re: SSL with Apache Nifi not working

Anishkumarv — Tue, 24 Jan 2017 06:59:44 GMT

@Matt