Member since
02-28-2022
17
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 52 | 06-21-2022 11:23 AM |
06-21-2022
11:23 AM
Thank you everyone for all the support and help. RCA: What I have observed, whenever we are performing PUT or Delete HTTP requests, the proxy in front of NiFi is intercepting and denying the request. Hence these requests are not even reaching our EC2 instance, and NIFI request.log is not capturing any PUT/DELETE HTTP requests. Hence on UI, we are getting 403 permissions Issues. Solution: We have enabled Put/ Delete HTTP requests from the proxy, and are now able to perform all the actions. Hence we can close this ticket.
... View more
06-14-2022
09:53 PM
@MattWho @gtorres @SAMSAL I am sharing my user logs. 2022-06-15 04:18:06,090 DEBUG [NiFi Web Server-18] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:18:06,090 DEBUG [NiFi Web Server-18] o.a.n.w.s.x509.X509CertificateExtractor No client certificate found in request.
2022-06-15 04:18:06,091 DEBUG [NiFi Web Server-18] o.a.n.w.s.j.k.StandardVerificationKeySelector Key Identifier [ec9c28d4-7330-48da-bdf5-dd398cd5b76f] Verification Keys Found [1]
2022-06-15 04:18:06,093 DEBUG [NiFi Web Server-143] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:18:06,093 DEBUG [NiFi Web Server-18] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,093 DEBUG [NiFi Web Server-143] o.a.n.w.s.x509.X509CertificateExtractor No client certificate found in request.
2022-06-15 04:18:06,093 DEBUG [NiFi Web Server-18] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,093 DEBUG [NiFi Web Server-143] o.a.n.w.s.j.k.StandardVerificationKeySelector Key Identifier [ec9c28d4-7330-48da-bdf5-dd398cd5b76f] Verification Keys Found [1]
2022-06-15 04:18:06,094 DEBUG [NiFi Web Server-143] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,094 DEBUG [NiFi Web Server-143] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,108 DEBUG [NiFi Web Server-128] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:18:06,108 DEBUG [NiFi Web Server-128] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntitiesChain - <********@Company.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=********.***.**.***>
2022-06-15 04:18:06,108 DEBUG [NiFi Web Server-128] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntityGroups - <>
2022-06-15 04:18:06,108 INFO [NiFi Web Server-128] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 10.204.230.155 [<********@Company.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=********.***.**.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=******.***>] GET https://********.***.**.***:8086/nifi-api/flow/status
2022-06-15 04:18:06,108 INFO [NiFi Web Server-128] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [********@Company.***] 10.204.230.155 GET https://********.***.**.***:8086/nifi-api/flow/status
2022-06-15 04:18:06,108 DEBUG [NiFi Web Server-128] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,108 DEBUG [NiFi Web Server-128] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,108 DEBUG [NiFi Web Server-136] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:18:06,108 DEBUG [NiFi Web Server-136] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntitiesChain - <********@Company.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=********.***.**.***>
2022-06-15 04:18:06,108 DEBUG [NiFi Web Server-136] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntityGroups - <>
2022-06-15 04:18:06,108 INFO [NiFi Web Server-136] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 10.204.230.155 [<********@Company.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=********.***.**.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=******.***>] GET https://********.***.**.***:8086/nifi-api/flow/process-groups/653f47d8-0181-1000-1d99-58b6a323962a
2022-06-15 04:18:06,109 INFO [NiFi Web Server-136] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [********@Company.***] 10.204.230.155 GET https://********.***.**.***:8086/nifi-api/flow/process-groups/653f47d8-0181-1000-1d99-58b6a323962a
2022-06-15 04:18:06,109 DEBUG [NiFi Web Server-136] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,109 DEBUG [NiFi Web Server-136] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,359 DEBUG [NiFi Web Server-136] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:18:06,359 DEBUG [NiFi Web Server-136] o.a.n.w.s.x509.X509CertificateExtractor No client certificate found in request.
2022-06-15 04:18:06,359 DEBUG [NiFi Web Server-128] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:18:06,360 DEBUG [NiFi Web Server-128] o.a.n.w.s.x509.X509CertificateExtractor No client certificate found in request.
2022-06-15 04:18:06,360 DEBUG [NiFi Web Server-136] o.a.n.w.s.j.k.StandardVerificationKeySelector Key Identifier [ec9c28d4-7330-48da-bdf5-dd398cd5b76f] Verification Keys Found [1]
2022-06-15 04:18:06,360 DEBUG [NiFi Web Server-128] o.a.n.w.s.j.k.StandardVerificationKeySelector Key Identifier [ec9c28d4-7330-48da-bdf5-dd398cd5b76f] Verification Keys Found [1]
2022-06-15 04:18:06,361 DEBUG [NiFi Web Server-143] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:18:06,361 DEBUG [NiFi Web Server-143] o.a.n.w.s.x509.X509CertificateExtractor No client certificate found in request.
2022-06-15 04:18:06,362 DEBUG [NiFi Web Server-143] o.a.n.w.s.j.k.StandardVerificationKeySelector Key Identifier [ec9c28d4-7330-48da-bdf5-dd398cd5b76f] Verification Keys Found [1]
2022-06-15 04:18:06,362 DEBUG [NiFi Web Server-136] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,362 DEBUG [NiFi Web Server-136] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,362 DEBUG [NiFi Web Server-128] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,362 DEBUG [NiFi Web Server-128] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,363 DEBUG [NiFi Web Server-143] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,363 DEBUG [NiFi Web Server-143] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,373 DEBUG [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:18:06,373 DEBUG [NiFi Web Server-20] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntitiesChain - <********@Company.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=********.***.**.***>
2022-06-15 04:18:06,373 DEBUG [NiFi Web Server-20] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntityGroups - <>
2022-06-15 04:18:06,373 INFO [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 10.204.230.155 [<********@Company.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=********.***.**.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=******.***>] GET https://********.***.**.***:8086/nifi-api/flow/current-user
2022-06-15 04:18:06,373 INFO [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [********@Company.***] 10.204.230.155 GET https://********.***.**.***:8086/nifi-api/flow/current-user
2022-06-15 04:18:06,373 DEBUG [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,373 DEBUG [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,378 DEBUG [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:18:06,378 DEBUG [NiFi Web Server-20] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntitiesChain - <********@Company.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=********.***.**.***>
2022-06-15 04:18:06,378 DEBUG [NiFi Web Server-20] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntityGroups - <>
2022-06-15 04:18:06,378 INFO [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 10.204.230.155 [<********@Company.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=********.***.**.***><C=US, ST=California, O=Company ***, OU=********.***.**.***, CN=******.***>] GET https://********.***.**.***:8086/nifi-api/flow/controller/bulletins
2022-06-15 04:18:06,378 INFO [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [********@Company.***] 10.204.230.155 GET https://********.***.**.***:8086/nifi-api/flow/controller/bulletins
2022-06-15 04:18:06,379 DEBUG [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:18:06,379 DEBUG [NiFi Web Server-20] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [********@Company.***]
2022-06-15 04:16:23,463 DEBUG [NiFi Web Server-120] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntityGroups - <>
2022-06-15 04:16:23,463 INFO [NiFi Web Server-120] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 10.204.230.155 [<xxxxxxxxx@company.com><C=US, ST=California, O=Company, OU=xxxxxxxxx, CN=zzzzzzzzzz.company.com><C=US, ST=California, O=Company, OU=xxxxxxxxx, CN=yyyyyyyy.company.com>] GET https://xxxxxxxxx.company.com:8086/nifi-api/policies/write/process-groups/653f47d8-0181-1000-1d99-58b6a323962a
2022-06-15 04:16:23,464 INFO [NiFi Web Server-120] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [xxxxxxxxx@company.com] 10.204.230.155 GET https://xxxxxxxxx.company.com:8086/nifi-api/policies/write/process-groups/653f47d8-0181-1000-1d99-58b6a323962a
2022-06-15 04:16:23,464 DEBUG [NiFi Web Server-120] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [xxxxxxxxx@company.com]
2022-06-15 04:16:23,464 DEBUG [NiFi Web Server-120] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [xxxxxxxxx@company.com]
2022-06-15 04:16:27,281 DEBUG [NiFi Web Server-21] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:16:27,281 DEBUG [NiFi Web Server-21] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntitiesChain - <xxxxxxxxx@company.com><C=US, ST=California, O=Company, OU=xxxxxxxxx, CN=zzzzzzzzzz.company.com>
2022-06-15 04:16:27,281 DEBUG [NiFi Web Server-21] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntityGroups - <>
2022-06-15 04:16:27,281 INFO [NiFi Web Server-21] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 10.204.230.155 [<xxxxxxxxx@company.com><C=US, ST=California, O=Company, OU=xxxxxxxxx, CN=zzzzzzzzzz.company.com><C=US, ST=California, O=Company, OU=xxxxxxxxx, CN=yyyyyyyy.company.com>] GET https://xxxxxxxxx.company.com:8086/nifi-api/policies/write/operation/process-groups/653f47d8-0181-1000-1d99-58b6a323962a
2022-06-15 04:16:27,282 INFO [NiFi Web Server-21] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [xxxxxxxxx@company.com] 10.204.230.155 GET https://xxxxxxxxx.company.com:8086/nifi-api/policies/write/operation/process-groups/653f47d8-0181-1000-1d99-58b6a323962a
2022-06-15 04:16:27,282 DEBUG [NiFi Web Server-21] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [xxxxxxxxx@company.com]
2022-06-15 04:16:27,282 DEBUG [NiFi Web Server-21] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [xxxxxxxxx@company.com]
2022-06-15 04:16:27,302 INFO [NiFi Web Server-21] o.a.n.w.a.c.ResourceNotFoundExceptionMapper org.apache.nifi.web.ResourceNotFoundException: Unable to find access policy for write on /operation/process-groups/653f47d8-0181-1000-1d99-58b6a323962a. Returning Not Found response.
2022-06-15 04:16:27,309 DEBUG [NiFi Web Server-21] o.a.n.w.a.c.ResourceNotFoundExceptionMapper
org.apache.nifi.web.ResourceNotFoundException: Unable to find access policy for write on /operation/process-groups/653f47d8-0181-1000-1d99-58b6a323962a
at org.apache.nifi.web.dao.impl.StandardPolicyBasedAuthorizerDAO.getAccessPolicy(StandardPolicyBasedAuthorizerDAO.java:201)
at org.apache.nifi.web.dao.impl.StandardPolicyBasedAuthorizerDAO$$FastClassBySpringCGLIB$$ea190383.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at org.apache.nifi.web.dao.impl.StandardPolicyBasedAuthorizerDAO$$EnhancerBySpringCGLIB$$be10ced9.getAccessPolicy(<generated>)
at org.apache.nifi.web.StandardNiFiServiceFacade.getAccessPolicy(StandardNiFiServiceFacade.java:4089)
at org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithReadLock(NiFiServiceFacadeLock.java:161)
at org.apache.nifi.web.NiFiServiceFacadeLock.getLock(NiFiServiceFacadeLock.java:120)
at jdk.internal.reflect.GeneratedMethodAccessor136.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$26e6223b.getAccessPolicy(<generated>)
at org.apache.nifi.web.api.AccessPolicyResource.getAccessPolicyForResource(AccessPolicyResource.java:162)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
at org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:58)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:58)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:121)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:94)
at org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:56)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:117)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:51)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.filter.ExceptionFilter.doFilter(ExceptionFilter.java:46)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:487)
at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:336)
at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:301)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.security.headers.StrictTransportSecurityFilter.doFilter(StrictTransportSecurityFilter.java:48)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.security.headers.XContentTypeOptionsFilter.doFilter(XContentTypeOptionsFilter.java:48)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.security.headers.XSSProtectionFilter.doFilter(XSSProtectionFilter.java:48)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.security.headers.ContentSecurityPolicyFilter.doFilter(ContentSecurityPolicyFilter.java:47)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.security.headers.XFrameOptionsFilter.doFilter(XFrameOptionsFilter.java:48)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-06-15 04:16:31,701 DEBUG [NiFi Web Server-120] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
2022-06-15 04:16:31,701 DEBUG [NiFi Web Server-120] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntitiesChain - <xxxxxxxxx@company.com><C=US, ST=California, O=Company, OU=xxxxxxxxx, CN=zzzzzzzzzz.company.com>
2022-06-15 04:16:31,701 DEBUG [NiFi Web Server-120] o.a.n.w.s.x509.X509AuthenticationFilter Raw X-ProxiedEntityGroups - <>
2022-06-15 04:16:31,701 INFO [NiFi Web Server-120] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 10.204.230.155 [<xxxxxxxxx@company.com><C=US, ST=California, O=Company, OU=xxxxxxxxx, CN=zzzzzzzzzz.company.com><C=US, ST=California, O=Company, OU=xxxxxxxxx, CN=yyyyyyyy.company.com>] POST https://xxxxxxxxx.company.com:8086/nifi-api/policies
2022-06-15 04:16:31,701 INFO [NiFi Web Server-120] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [xxxxxxxxx@company.com] 10.204.230.155 POST https://xxxxxxxxx.company.com:8086/nifi-api/policies user.logs Here we have observed that whenever NIFI tries to get status or fetch the current user, we get below mentioned logs, weherNiFiAuthenticationFilter Authenticating sometimes gets the user and sometimes log with null: 10.204.230.155 GET https://xxxxxxxxx.company.com:8086/nifi-api/flow/status 2022-06-15 04:14:57,583 DEBUG [NiFi Web Server-21] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [xxxxxxxxx@company.com] 2022-06-15 04:14:57,583 DEBUG [NiFi Web Server-25] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [xxxxxxxxx@company.com] 2022-06-15 04:14:57,583 DEBUG [NiFi Web Server-21] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [xxxxxxxxx@company.com] 2022-06-15 04:14:57,583 DEBUG [NiFi Web Server-25] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [xxxxxxxxx@company.com] 2022-06-15 04:14:57,818 DEBUG [NiFi Web Server-120] o.a.n.w.s.NiFiAuthenticationFilter Authenticating [null]
... View more
06-12-2022
12:50 AM
@MattWho Thank you for pointing out logback.xml, now am able to generate the users.log.
... View more
06-10-2022
12:08 PM
@MattWho 1. Have you tried using a different web browser like Firefox? I am unable to log in via Mozilla, my SAML service is not working for callbacks to NIFI. However, used Safari and Chrome. 2. Have you tried opening your browser's Developer tools and inspecting the actual rest-api call that was made when you attempt the various actions that fail from with the NiFi UI? Yes, I did it. I am sharing details for few of the actions: Request URL: https://xxxxxxxx/nifi-api/policies/43dfca36-0181-1000-ffff-ffff90447006 Request Method: PUT Status Code: 403 Payload: {"revision":{"clientId":"4806d375-0181-1000-e24c-c2e2244d8578","version":0},"disconnectedNodeAcknowledged":false,"component":{"id":"43dfca36-0181-1000-ffff-ffff90447006","users":[{"revision":{"version":0},"id":"5c72646f-a9cb-3239-9450-2511231b004e","permissions":{"canRead":true,"canWrite":true},"component":{"id":"5c72646f-a9cb-3239-9450-2511231b004e","identity":"xxxx@xxx.com","configurable":true}}],"userGroups":[]}} Request URL: https://xxxxxxxxx/nifi-api/flow/process-groups/439aeba6-0181-1000-fabf-28c9c87d5ce8/controller-services Request Method: PUT Status Code: 403 Payload: {"id":"439aeba6-0181-1000-fabf-28c9c87d5ce8","state":"ENABLED","disconnectedNodeAcknowledged":false} https://xxxxxxx/nifi-api/access/logout Request Method: DELETE Status Code: 403 3. Are you going through a proxy or load balancer (is it configured to use sticky sessions?)? Yes, I am using a proxy and have configured sticky sessions. 4. Which Browser and version are you using? Safari: User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15 Chrome: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Safari/537.36 For both, the browser NIFI adds Mozilla/5.0 in user-agent 5. Have you tried clearing your browser cache? - Yes I tried clearing my cache, It exhibits the same behavior. 6. Does the same behavior exist using an incognito window in your browser? I have tried on Safari, and Chrome. However, I have used Safari private window as well. It exhibits the same behavior. However unable to Logging using Chrome- incognito window. 7. What java version is your NiFi using? - JDK-11.0.14.9.2 In addition to these, I would like to bring you to notice that, I have tried NIFI- 1.16.0 on AWS with a single node setup, using single-user-authorizer. It works absolutely fine. And I am able to use NIFI smoothly. Once I tried using managed-authorizer with SAML on 3 node cluster. Then I am facing all these permission issues. However, I have an absolutely similar setup with 3 node cluster for NIFI-1.150 (not on AWS, it is on the private cloud). There I am able to get access to the main root process policy. And able to access processors can create processors, but can not delete processors, Logout and add policy for other users or myself.
... View more
06-10-2022
10:55 AM
@MattWho I have tried to click on that key and add my user to the policies. But still, it gives me 403 insufficient permissions : Please find the Request/Response Details: Request URL: https://xxxxxxxxxxxxxxx/nifi-api/policies/43f0b681-0181-1000-ffff-ffffc15af0d7 Request Method: PUT Status Code: 403 Payload: ion":{"clientId":"4eb9cca3-0181-1000-5ccc-79d4aacc5540","version":0},"disconnectedNodeAcknowledged":false,"component":{"id":"43f0b681-0181-1000-ffff-ffffc15af0d7","users":[{"revision":{"version":0},"id":"5c72646f-a9cb-3239-9450-2511231b004e","permissions":{"canRead":true,"canWrite":true},"component":{"id":"5c72646f-a9cb-3239-9450-2511231b004e","identity":"*****@*******.com","configurable":true}}],"userGroups":[]}} Response: <html> <head><title>403 Forbidden</title></head> <body> <center><h1>403 Forbidden</h1></center> <hr><center>Apple</center> </body> </html> If I am trying to add any policy to my user, it is giving me insufficient permissions. Thank you!!!
... View more
06-10-2022
10:33 AM
Hi @MattWho, Thank you for replying, it's my bad, I have missed posting my complete authorizers.xml which have <authorizer>
<identifier>managed-authorizer</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="Access Policy Provider">file-access-policy-provider</property>
</authorizer>
... View more
06-08-2022
08:00 PM
HI @gtorres, I am not able to see any 403 activity even getting logged in my servers. I have not found any such 403 request/response calls in App logs, user logs, Request logs. Whenever I am getting 403 while performing any actions, it is not getting captured in any log file (App, User, Request). https://community.cloudera.com/t5/Support-Questions/Unable-to-seed-access-policy-on-NIFI-1-16-0-and-User-logs/td-p/345246
... View more
06-08-2022
07:54 PM
HI Team, System Env Details: I am trying setup NIFI on AWS 3 node cluster NIFI version: 1.16.0 Primarily I tried to setup 1 node with single-user-provider on AWS with NIFI-1.16.0 and able to perform each and every action. It is working fine. Then I tried to setup 3 node cluster on AWS, What went well. 1. Configured zookeeper in nifi.properties and Statemanagement.xml and able to formulate a cluster and cluster coordinator setup is successful. 2. using nifi.security.user.authorizer=managed-authorizer 3. Successfully able to integrate with SAML and able to login as a Admin user. What went wrong: 1. Except User-logs, all other logs are getting generated (app, request, bootstrap) 2. When I have logged in as an Initial Admin Identity, Ideally I should be able to add users and policy to them. However, I am getting insufficient permissions while performing any action. (add policy, delete used, edit user, logout etc...) Any help would be highly appreciated. For my detailed set kindly visit my post: https://community.cloudera.com/t5/Support-Questions/NIFI-1-16-0-insufficient-permissions-while-performing-any/td-p/345150
... View more
Labels:
- Labels:
-
Apache NiFi
06-08-2022
08:18 AM
These are the list of permissions I have to ADMIN User: {
"identity": "xxxx@xxx.com",
"anonymous": false,
"provenancePermissions": {
"canRead": false,
"canWrite": false
},
"countersPermissions": {
"canRead": false,
"canWrite": false
},
"tenantsPermissions": {
"canRead": true,
"canWrite": true
},
"controllerPermissions": {
"canRead": true,
"canWrite": true
},
"policiesPermissions": {
"canRead": true,
"canWrite": true
},
"systemPermissions": {
"canRead": false,
"canWrite": false
},
"parameterContextPermissions": {
"canRead": true,
"canWrite": true
},
"restrictedComponentsPermissions": {
"canRead": false,
"canWrite": true
},
"componentRestrictionPermissions": [
{
"requiredPermission": {
"id": "read-distributed-filesystem",
"label": "read distributed filesystem"
},
"permissions": {
"canRead": false,
"canWrite": true
}
},
{
"requiredPermission": {
"id": "access-keytab",
"label": "access keytab"
},
"permissions": {
"canRead": false,
"canWrite": true
}
},
{
"requiredPermission": {
"id": "export-nifi-details",
"label": "export nifi details"
},
"permissions": {
"canRead": false,
"canWrite": true
}
},
{
"requiredPermission": {
"id": "read-filesystem",
"label": "read filesystem"
},
"permissions": {
"canRead": false,
"canWrite": true
}
},
{
"requiredPermission": {
"id": "access-environment-credentials",
"label": "access environment credentials"
},
"permissions": {
"canRead": false,
"canWrite": true
}
},
{
"requiredPermission": {
"id": "execute-code",
"label": "execute code"
},
"permissions": {
"canRead": false,
"canWrite": true
}
},
{
"requiredPermission": {
"id": "access-ticket-cache",
"label": "access ticket cache"
},
"permissions": {
"canRead": false,
"canWrite": true
}
},
{
"requiredPermission": {
"id": "write-filesystem",
"label": "write filesystem"
},
"permissions": {
"canRead": false,
"canWrite": true
}
},
{
"requiredPermission": {
"id": "write-distributed-filesystem",
"label": "write distributed filesystem"
},
"permissions": {
"canRead": false,
"canWrite": true
}
}
],
"canVersionFlows": false
}
... View more
06-08-2022
07:50 AM
Hi @araujo, Have you experience it ? Request your help. Thanks
... View more
06-08-2022
07:46 AM
I have restarted NIFI by deleting "authorisations.xml " and "users.xml" My authorisations.xml: <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizations>
<policies>
<policy identifier="f99bccd1-a30e-3e4a-98a2-dbc708edc67f" resource="/flow" action="R">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
</policy>
<policy identifier="91b40b98-fd71-37df-b47f-0b6bfc9495a3" resource="/data/process-groups/040813bd-98b2-35d7-9202-cc93f4873b91" action="R">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
<user identifier="2e8a3365-86df-3c2d-8614-7cf2c61cc5d1"/>
<user identifier="a98e60ec-bb80-35c6-bd31-1d1c7562a5a8"/>
<user identifier="6d337594-bfba-3a95-88a7-01ec124f0bdb"/>
</policy>
<policy identifier="b855b111-a013-30b7-828f-8c97c4dd5451" resource="/data/process-groups/040813bd-98b2-35d7-9202-cc93f4873b91" action="W">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
<user identifier="2e8a3365-86df-3c2d-8614-7cf2c61cc5d1"/>
<user identifier="a98e60ec-bb80-35c6-bd31-1d1c7562a5a8"/>
<user identifier="6d337594-bfba-3a95-88a7-01ec124f0bdb"/>
</policy>
<policy identifier="0e420cf9-3421-30ec-aab3-897e5bf63106" resource="/process-groups/040813bd-98b2-35d7-9202-cc93f4873b91" action="R">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
</policy>
<policy identifier="3e78b62e-721b-336f-a898-70534c5aa1a9" resource="/process-groups/040813bd-98b2-35d7-9202-cc93f4873b91" action="W">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
</policy>
<policy identifier="b8775bd4-704a-34c6-987b-84f2daf7a515" resource="/restricted-components" action="W">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
</policy>
<policy identifier="627410be-1717-35b4-a06f-e9362b89e0b7" resource="/tenants" action="R">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
</policy>
<policy identifier="15e4e0bd-cb28-34fd-8587-f8d15162cba5" resource="/tenants" action="W">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
</policy>
<policy identifier="ff96062a-fa99-36dc-9942-0f6442ae7212" resource="/policies" action="R">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
</policy>
<policy identifier="ad99ea98-3af6-3561-ae27-5bf09e1d969d" resource="/policies" action="W">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
</policy>
<policy identifier="2e1015cb-0fed-3005-8e0d-722311f21a03" resource="/controller" action="R">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
</policy>
<policy identifier="c6322e6c-4cc1-3bcc-91b3-2ed2111674cf" resource="/controller" action="W">
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
</policy>
<policy identifier="287edf48-da72-359b-8f61-da5d4c45a270" resource="/proxy" action="W">
<user identifier="2e8a3365-86df-3c2d-8614-7cf2c61cc5d1"/>
<user identifier="a98e60ec-bb80-35c6-bd31-1d1c7562a5a8"/>
<user identifier="6d337594-bfba-3a95-88a7-01ec124f0bdb"/>
</policy>
</policies>
</authorizations> Here my user Identifier is: <user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/> But still, I am unable to perform any actions as mentioned in my Post earlier. I have also observed, that it shows unauthorised for main process group:
... View more
06-07-2022
10:46 PM
Certificates are for cluster Nodes. As I am using SAML, so I have set my email DI as Initial admin property. And while authenticating calls are redirected to my SAML service and they are authenticating my user. Please let me know if I am missing something.
... View more
06-07-2022
10:05 AM
HI Team, System Env Details: I am trying setup NIFI on AWS 3 node cluster NIFI version: 1.16.0 authorisers.xml :
<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
<property name="Initial User Identity 1">xxx@xxx.com</property>
<property name="Initial User Identity 2">C=US, ST=xxx, O=xxx, OU=xxxx, CN=%%cn_user_identity_1%%</property>
<property name="Initial User Identity 3">C=US, ST=xxx, O=xxx, OU=xxxx, CN=%%cn_user_identity_2%%</property>
<property name="Initial User Identity 4">C=US, ST=xxxxx, O=xxxx, OU=xxxx, CN=%%cn_user_identity_3%%</property>
</userGroupProvider>
<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Initial Admin Identity">xxxx@xxxxx.com</property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1">C=US, ST=xxx, O=xxx, OU=xxx, CN=%%cn_node_identity_1%%</property>
<property name="Node Identity 2">C=US, ST=xxx, O=xx, OU=xxx, CN=%%cn_node_identity_2%%</property>
<property name="Node Identity 3">C=US, ST=xxxx, O=xxxxx, OU=xxxx, CN=%%cn_node_identity_3%%</property>
<property name="Node Group"></property>
</accessPolicyProvider>
nifi.properties
# Core Properties #
nifi.flow.configuration.file=./conf/flow.xml.gz
nifi.flow.configuration.json.file=./conf/flow.json.gz
nifi.flow.configuration.archive.enabled=true
nifi.flow.configuration.archive.dir=./conf/archive/
nifi.flow.configuration.archive.max.time=30 days
nifi.flow.configuration.archive.max.storage=500 MB
nifi.flow.configuration.archive.max.count=
nifi.flowcontroller.autoResumeState=true
nifi.flowcontroller.graceful.shutdown.period=10 sec
nifi.flowservice.writedelay.interval=500 ms
nifi.administrative.yield.duration=30 sec
# If a component has no work to do (is "bored"), how long should we wait before checking again for work?
nifi.bored.yield.duration=10 millis
nifi.queue.backpressure.count=10000
nifi.queue.backpressure.size=1 GB
nifi.authorizer.configuration.file=./conf/authorizers.xml
nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
nifi.templates.directory=./conf/templates
nifi.ui.banner.text=
nifi.ui.autorefresh.interval=30 sec
nifi.nar.library.directory=./lib
nifi.nar.library.autoload.directory=./extensions
nifi.nar.working.directory=./work/nar/
nifi.documentation.working.directory=./work/docs/components
####################
# State Management #
####################
nifi.state.management.configuration.file=./conf/state-management.xml
# The ID of the local state provider
nifi.state.management.provider.local=local-provider
# The ID of the cluster-wide state provider. This will be ignored if NiFi is not clustered but must be populated if running in a cluster.
nifi.state.management.provider.cluster=zk-provider
# Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server
nifi.state.management.embedded.zookeeper.start=false
# Properties file that provides the ZooKeeper properties to use if <nifi.state.management.embedded.zookeeper.start> is set to true
nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties
# H2 Settings
nifi.database.directory=%%base_path%%/database_repository
nifi.h2.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
# Repository Encryption properties override individual repository implementation properties
nifi.repository.encryption.protocol.version=
nifi.repository.encryption.key.id=
nifi.repository.encryption.key.provider=
nifi.repository.encryption.key.provider.keystore.location=
nifi.repository.encryption.key.provider.keystore.password=
# FlowFile Repository
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.wal.implementation=org.apache.nifi.wali.SequentialAccessWriteAheadLog
nifi.flowfile.repository.directory=%%base_path%%/flowfile_repository
nifi.flowfile.repository.checkpoint.interval=2 mins
nifi.flowfile.repository.always.sync=false
nifi.flowfile.repository.retain.orphaned.flowfiles=true
nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000
# Content Repository
nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository
nifi.content.claim.max.appendable.size=1 MB
nifi.content.repository.directory.default=%%base_path%%/content_repository
nifi.content.repository.archive.max.retention.period=12 hours
nifi.content.repository.archive.max.usage.percentage=50%
nifi.content.repository.archive.enabled=true
nifi.content.repository.always.sync=false
nifi.content.viewer.url=../nifi-content-viewer/
# Provenance Repository Properties
nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
# Persistent Provenance Repository Properties
nifi.provenance.repository.directory.default=%%base_path%%/provenance_repository
nifi.provenance.repository.max.storage.time=24 hours
nifi.provenance.repository.max.storage.size=1 GB
nifi.provenance.repository.rollover.time=30 secs
nifi.provenance.repository.rollover.size=100 MB
nifi.provenance.repository.query.threads=2
nifi.provenance.repository.index.threads=2
nifi.provenance.repository.compress.on.rollover=true
nifi.provenance.repository.always.sync=false
# Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are:
# EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, Relationship, Details
nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship
# FlowFile Attributes that should be indexed and made searchable. Some examples to consider are filename, uuid, mime.type
nifi.provenance.repository.indexed.attributes=
# Large values for the shard size will result in more Java heap usage when searching the Provenance Repository
# but should provide better performance
nifi.provenance.repository.index.shard.size=500 MB
# Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from
# the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved.
nifi.provenance.repository.max.attribute.length=65536
nifi.provenance.repository.concurrent.merge.threads=2
# Volatile Provenance Respository Properties
nifi.provenance.repository.buffer.size=100000
# Component and Node Status History Repository
nifi.components.status.repository.implementation=org.apache.nifi.controller.status.history.VolatileComponentStatusRepository
# Volatile Status History Repository Properties
nifi.components.status.repository.buffer.size=1440
nifi.components.status.snapshot.frequency=1 min
# QuestDB Status History Repository Properties
nifi.status.repository.questdb.persist.node.days=14
nifi.status.repository.questdb.persist.component.days=3
nifi.status.repository.questdb.persist.location=%%base_path%%/status_repository
# Site to Site properties
nifi.remote.input.host=%%host_name%%
nifi.remote.input.secure=true
nifi.remote.input.socket.port=10443
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs
# web properties #
#############################################
# For security, NiFi will present the UI on 127.0.0.1 and only be accessible through this loopback interface.
# Be aware that changing these properties may affect how your instance can be accessed without any restriction.
# We recommend configuring HTTPS instead. The administrators guide provides instructions on how to do this.
nifi.web.http.host=
nifi.web.http.port=
nifi.web.http.network.interface.default=
#############################################
nifi.web.https.host=%%host_name%%
nifi.web.https.port=8086
nifi.web.https.network.interface.default=
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
nifi.web.max.header.size=16 KB
nifi.web.proxy.context.path=
nifi.web.proxy.host=%%proxy_hostname%%:443
nifi.web.max.content.size=
nifi.web.max.requests.per.second=30000
nifi.web.max.access.token.requests.per.second=25
nifi.web.request.timeout=60 secs
nifi.web.request.ip.whitelist=
nifi.web.should.send.server.version=true
nifi.web.request.log.format=%{client}a - %u %t "%r" %s %O "%{Referer}i" "%{User-Agent}i"
# Include or Exclude TLS Cipher Suites for HTTPS
nifi.web.https.ciphersuites.include=
nifi.web.https.ciphersuites.exclude=
# security properties #
nifi.sensitive.props.key=***************
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=NIFI_PBKDF2_AES_GCM_256
nifi.sensitive.props.additional.keys=
nifi.security.autoreload.enabled=false
nifi.security.autoreload.interval=10 secs
nifi.security.keystore=%%base_path%%/certs/SSL/%%host_name%%.keystore
nifi.security.keystoreType=jks
nifi.security.keystorePasswd=**********
nifi.security.keyPasswd=*********
nifi.security.truststore=%%base_path%%/certs/SSL/truststore.jks
nifi.security.truststoreType=jks
nifi.security.truststorePasswd=***********
nifi.security.user.authorizer=managed-authorizer
nifi.security.allow.anonymous.authentication=false
nifi.security.user.login.identity.provider=
nifi.security.user.jws.key.rotation.period=
nifi.security.ocsp.responder.url=
nifi.security.ocsp.responder.certificate=
# OpenId Connect SSO Properties #
nifi.security.user.oidc.discovery.url=
nifi.security.user.oidc.connect.timeout=5 secs
nifi.security.user.oidc.read.timeout=5 secs
nifi.security.user.oidc.client.id=
nifi.security.user.oidc.client.secret=
nifi.security.user.oidc.preferred.jwsalgorithm=
nifi.security.user.oidc.additional.scopes=
nifi.security.user.oidc.claim.identifying.user=
nifi.security.user.oidc.fallback.claims.identifying.user=
nifi.security.user.oidc.truststore.strategy=JDK
# Apache Knox SSO Properties #
nifi.security.user.knox.url=
nifi.security.user.knox.publicKey=
nifi.security.user.knox.cookieName=hadoop-jwt
nifi.security.user.knox.audiences=
# SAML Properties #
nifi.security.user.saml.idp.metadata.url=file://%%base_path%%/certs/IDMS/idms-xxxxx.xml
nifi.security.user.saml.sp.entity.id=xxxxxxx
nifi.security.user.saml.identity.attribute.name=email
nifi.security.user.saml.group.attribute.name=
nifi.security.user.saml.metadata.signing.enabled=false
nifi.security.user.saml.request.signing.enabled=false
nifi.security.user.saml.want.assertions.signed=true
nifi.security.user.saml.signature.algorithm=http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
nifi.security.user.saml.signature.digest.algorithm=http://www.w3.org/2001/04/xmlenc#sha256
nifi.security.user.saml.message.logging.enabled=true
nifi.security.user.saml.authentication.expiration=8 hours
nifi.security.user.saml.single.logout.enabled=false
nifi.security.user.saml.http.client.truststore.strategy=JDK
nifi.security.user.saml.http.client.connect.timeout=30 secs
nifi.security.user.saml.http.client.read.timeout=30 secs
# Identity Mapping Properties #
# These properties allow normalizing user identities such that identities coming from different identity providers
# (certificates, LDAP, Kerberos) can be treated the same internally in NiFi. The following example demonstrates normalizing
# DNs from certificates and principals from Kerberos into a common identity string:
#
# nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$
# nifi.security.identity.mapping.value.dn=$1@$2
# nifi.security.identity.mapping.transform.dn=NONE
# nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$
# nifi.security.identity.mapping.value.kerb=$1@$2
# nifi.security.identity.mapping.transform.kerb=UPPER
# Group Mapping Properties #
# These properties allow normalizing group names coming from external sources like LDAP. The following example
# lowercases any group name.
#
# nifi.security.group.mapping.pattern.anygroup=^(.*)$
# nifi.security.group.mapping.value.anygroup=$1
# nifi.security.group.mapping.transform.anygroup=LOWER
# Listener Bootstrap properties #
# This property defines the port used to listen for communications from NiFi Bootstrap. If this property
# is missing, empty, or 0, a random ephemeral port is used.
nifi.listener.bootstrap.port=0
# cluster common properties (all nodes must have same values) #
nifi.cluster.protocol.heartbeat.interval=5 sec
nifi.cluster.protocol.heartbeat.missable.max=8
nifi.cluster.protocol.is.secure=true
# cluster node properties (only configure for cluster nodes) #
nifi.cluster.is.node=true
nifi.cluster.node.address=%%host_name%%
nifi.cluster.node.protocol.port=11443
nifi.cluster.node.protocol.max.threads=50
nifi.cluster.node.event.history.size=25
nifi.cluster.node.connection.timeout=10 sec
nifi.cluster.node.read.timeout=10 sec
nifi.cluster.node.max.concurrent.requests=100
nifi.cluster.firewall.file=
nifi.cluster.flow.election.max.wait.time=1 mins
nifi.cluster.flow.election.max.candidates=
# cluster load balancing properties #
nifi.cluster.load.balance.host=
nifi.cluster.load.balance.port=6342
nifi.cluster.load.balance.connections.per.node=4
nifi.cluster.load.balance.max.thread.count=8
nifi.cluster.load.balance.comms.timeout=30 sec
# zookeeper properties, used for cluster management #
nifi.zookeeper.connect.string=%%zookeeper_connection_strings%%
nifi.zookeeper.connect.timeout=10 secs
nifi.zookeeper.session.timeout=10 secs
nifi.zookeeper.root.node=%%base_path%%/nifi
nifi.zookeeper.client.secure=true
nifi.zookeeper.security.keystore=
nifi.zookeeper.security.keystoreType=
nifi.zookeeper.security.keystorePasswd=
nifi.zookeeper.security.truststore=
nifi.zookeeper.security.truststoreType=
nifi.zookeeper.security.truststorePasswd=
nifi.zookeeper.jute.maxbuffer=
# Zookeeper properties for the authentication scheme used when creating acls on znodes used for cluster management
# Values supported for nifi.zookeeper.auth.type are "default", which will apply world/anyone rights on znodes
# and "sasl" which will give rights to the sasl/kerberos identity used to authenticate the nifi node
# The identity is determined using the value in nifi.kerberos.service.principal and the removeHostFromPrincipal
# and removeRealmFromPrincipal values (which should align with the kerberos.removeHostFromPrincipal and kerberos.removeRealmFromPrincipal
# values configured on the zookeeper server).
nifi.zookeeper.auth.type=
nifi.zookeeper.kerberos.removeHostFromPrincipal=
nifi.zookeeper.kerberos.removeRealmFromPrincipal=
# kerberos #
nifi.kerberos.krb5.file=
# kerberos service principal #
nifi.kerberos.service.principal=
nifi.kerberos.service.keytab.location=
# kerberos spnego principal #
nifi.kerberos.spnego.principal=
nifi.kerberos.spnego.keytab.location=
nifi.kerberos.spnego.authentication.expiration=12 hours
# external properties files for variable registry
# supports a comma delimited list of file locations
nifi.variable.registry.properties=
# analytics properties #
nifi.analytics.predict.enabled=false
nifi.analytics.predict.interval=3 mins
nifi.analytics.query.interval=5 mins
nifi.analytics.connection.model.implementation=org.apache.nifi.controller.status.analytics.models.OrdinaryLeastSquares
nifi.analytics.connection.model.score.name=rSquared
nifi.analytics.connection.model.score.threshold=.90
# runtime monitoring properties
nifi.monitor.long.running.task.schedule=
nifi.monitor.long.running.task.threshold=
# Create automatic diagnostics when stopping/restarting NiFi.
# Enable automatic diagnostic at shutdown.
nifi.diagnostics.on.shutdown.enabled=false
# Include verbose diagnostic information.
nifi.diagnostics.on.shutdown.verbose=false
# The location of the diagnostics folder.
nifi.diagnostics.on.shutdown.directory=./diagnostics
# The maximum number of files permitted in the directory. If the limit is exceeded, the oldest files are deleted.
nifi.diagnostics.on.shutdown.max.filecount=10
# The diagnostics folder's maximum permitted size in bytes. If the limit is exceeded, the oldest files are deleted.
nifi.diagnostics.on.shutdown.max.directory.size=10 MB
Users.xml:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
<groups>
<group identifier="39903356-0181-1000-0000-000047f790c3" name="Admin">
<user identifier="3989a18a-0181-1000-0000-0000280a357e"/>
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e"/>
<user identifier="6d337594-bfba-3a95-88a7-01ec124f0bdb"/>
<user identifier="a98e60ec-bb80-35c6-bd31-1d1c7562a5a8"/>
<user identifier="2e8a3365-86df-3c2d-8614-7cf2c61cc5d1"/>
</group>
</groups>
<users>
<user identifier="2e8a3365-86df-3c2d-8614-7cf2c61cc5d1" identity="C=US, ST=xxx, O=xxx, OU=xxx, CN=xxxx"/>
<user identifier="a98e60ec-bb80-35c6-bd31-1d1c7562a5a8" identity="C=US, ST=xxx, O=xxx, OU=xxx, CN=xxx"/>
<user identifier="5c72646f-a9cb-3239-9450-2511231b004e" identity="xxx@xxx.com"/>
<user identifier="6d337594-bfba-3a95-88a7-01ec124f0bdb" identity="C=US, ST=xxx, O=xxx, OU=xxxx, CN=xxxxx"/>
<user identifier="3989a18a-0181-1000-0000-0000280a357e" identity="abc@abc.com"/>
</users>
</tenants>
I am using SAML and successfully able to login as an Initial Admin Identity. However after login I can see my UI as seen in image, where all he processors are disabled for me. And if I perform any action like logout or add/edit user, add policy.... I get below error message: I have similar setup with NIFI version 1.10.0, and it is working fine. But with Nifi-1.15.0 or Nifi-1.16.0 I am unable to perform any action. Any help would be highly appreciated.
... View more
- Tags:
- NiFi
- Permissions
Labels:
- Labels:
-
Apache NiFi
04-10-2022
10:32 PM
Hi, Nifi is not capable of spinning up a new node if one goes down, some of the threads in community says, autoscaling is possible in AWS .
AWS Auto Scaling monitors your applications and automatically adjusts capacity to maintain a steady, predictable predefined number of a node if a specifically configured alert is triggered eg Bad health or lost heartbeat.
We have a setup in our 3 fixed node cluster. To established authorised nodes communication within the cluster we need to provide host names in authorizers.xml file. So I need to know how we can autoscale our NIFI clusters, where we need to update our authorizers.xml file in each and every instance. Any help would be highly appreciated. Thanks
Abhishek
... View more
- Tags:
- aws
- NiFi
- node cluster
Labels:
- Labels:
-
Apache NiFi
03-01-2022
10:19 PM
@araujo , # SAML Properties # nifi.security.user.saml.idp.metadata.url=file:///pathtoIPDMetaDataFile/ nifi.security.user.saml.sp.entity.id =xxxxx nifi.security.user.saml.identity.attribute.name = email nifi.security.user.saml.group.attribute.name = nifi.security.user.saml.metadata.signing.enabled = false nifi.security.user.saml.request.signing.enabled = false nifi.security.user.saml.want.assertions.signed = true nifi.security.user.saml.signature.algorithm = http://www.w3.org/2001/04/xmldsig-more#rsa-sha256 nifi.security.user.saml.signature.digest.algorithm = http://www.w3.org/2001/04/xmlenc#sha256 nifi.security.user.saml.message.logging.enabled = true nifi.security.user.saml.authentication.expiration = 8 hours nifi.security.user.saml.single.logout.enabled = false nifi.security.user.saml.http.client.truststore.strategy = JDK nifi.security.user.saml.http.client.connect.timeout = 30 secs nifi.security.user.saml.http.client.read.timeout = 30 secs Abhishek
... View more
02-28-2022
02:52 AM
HI @araujo, In our authorizers.xml, we have below mentioned entry: <userGroupProvider> <identifier>file-user-group-provider</identifier> <class>org.apache.nifi.authorization.FileUserGroupProvider</class> <property name="Users File">./conf/users.xml</property> <property name="Legacy Authorized Users File"></property> <property name="Initial User Identity 1">xxxx@xxx.com</property> <property name="Initial User Identity 2">servernode1 XXXXXX</property> <property name="Initial User Identity 3">servernode2 XXXXXX</property> <property name="Initial User Identity 4">servernode3 XXXXXX</property> </userGroupProvider> <accessPolicyProvider> <identifier>file-access-policy-provider</identifier> <class>org.apache.nifi.authorization.FileAccessPolicyProvider</class> <property name="User Group Provider">file-user-group-provider</property> <property name="Authorizations File">./conf/authorizations.xml</property> <property name="Initial Admin Identity">xxxxxxxx@xxx.com</property> <property name="Legacy Authorized Users File"></property> <property name="Node Identity 1">servernode1 XXXXXX</property> <property name="Node Identity 2">servernode2 xxxxxx</property> <property name="Node Identity 3">servernode3 xxxxx</property> <property name="Node Group"></property> </accessPolicyProvider> <authorizer> <identifier>managed-authorizer</identifier> <class>org.apache.nifi.authorization.StandardManagedAuthorizer</class> <property name="Access Policy Provider">file-access-policy-provider</property> </authorizer> ############################################################### nifi-app.log ########################### Server node 1: 2022-02-28 10:42:37,897 ERROR [NiFi Web Server-162] o.apache.nifi.web.api.SAMLAccessResource The RelayState value returned by the SAML IDP does not match the stored state. Unable to continue login process. ########################## Server node 2: 2022-02-28 10:42:27,550 WARN [NiFi Web Server-151] o.apache.nifi.web.api.SAMLAccessResource The login request identifier was not found in the request. Unable to continue. ######################## Server node 3: This is the node where we see the SAML request Logs: 2022-02-28 10:42:28,269 DEBUG [NiFi Web Server-217] org.apache.velocity.loader ResourceManager: found /templates/saml2-post-binding.vm with loader org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader 2022-02-28 10:42:28,270 DEBUG [NiFi Web Server-217] org.apache.velocity.loader ResourceManager: found /templates/add-html-head-content.vm with loader org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader 2022-02-28 10:42:28,271 DEBUG [NiFi Web Server-217] org.apache.velocity.loader ResourceManager: found /templates/add-html-body-content.vm with loader org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader 2022-02-28 10:42:28,273 DEBUG [NiFi Web Server-217] PROTOCOL_MESSAGE <?xml version="1.0" encoding="UTF-8"?> <saml2p:AuthnRequest AssertionConsumerServiceURL="https://xxxxxxxx/nifi-api/access/saml/login/consumer" Destination="xxxxxxxxx" ForceAuthn="false" ID="xxxxx" IsPassive="false" IssueInstant="2022-02-28T10:42:28.261Z" ProtocolBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Version="2.0" xmlns:saml2p="urn:oasis:names:tc:SAML:2.0:protocol"> <saml2:Issuer xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion">xxxxxxx</saml2:Issuer> </saml2p:AuthnRequest>
... View more
