Member since
08-01-2022
5
Posts
0
Kudos Received
0
Solutions
09-22-2022
07:55 AM
@MattWho Hi Matt, the registry is also version 1.15.3. Some more context: the cluster contains dozens of other process groups that have been managed in the same way and versioned on the same registry for months. This is the first time we experience this issue. The component with UUID a8db3982-1350-1b8b-ffff-fffff988699d is the "freezed" process group itself: This is its configuration: And its associated controller services: I still think that the fastest and safest solution is to delete and re-deploy. Is it possible? S
... View more
09-21-2022
03:01 AM
Hi Matt, Thank you for your help. I'lll try to give you more context about the issue. Yes, all the nodes of the cluster run the same version: 1.15.3 and the same libs. Unfortunately, shutting down the entire cluster is not an option because it is a production environment and it receives live streaming data we cannot afford to lose. We operate the environment this way: we create the process groups, flows and configurations on a staging cluster (3 nodes, same NiFi version 1.15.3). We commit the changes to the registry Then we pull them to the production cluster (11 nodes) from the registry. This is why I'm sure that the configurations stored in the registry are valid and working: the same process group on the staging cluster has no issues. Are you saying that if you access the NiFi UI from a different node in your 11 node cluster, this process groups renders differently? No, the process group renders the same on all nodes, but every attempt to do anything on it results in the same error message: We cannot edit it, delete, change version, detach from version control, not even move it. It always returns the same error, albeit no modification is present locally. In fact, the version control menu seems to "believe" that there are local changes to the PG: But then, if I select "Show local changes", nothing is shown, as expected: Same if I select "Revert local changes": My assumption is that the process group definition in the local flow file is corrupted or not in sync with the version control of the registry. So I think that the best solution is to just force-delete the PG and the re-create it. Is there a way to do this? Thank you S
... View more
09-20-2022
08:51 AM
Hello everyone, I'm running a 11-nodes NiFi 1.15.3 cluster. One of the process groups is versioned on NiFi Registry and for some reason the local flowfile does not reflect the versioned configuration, so now the process group is stuck: I cannot do anything on it, not even moving it on the canvas because it always return an error: Node XXXXXXXXX is unable to fulfill this request due to: [15, xxxxx-xxxxx-xxxxxx] is not the most up-to-date revision. This component appears to have been modified The local configuration shows no changes, and nothing I tried so far worked (deleting the flow file, restarting the cluster node, etc). so I just want to delete the process group and deploy it again from the registry, but the web interface won't let me, throwing the same error. Is there a way to force the deletion the process group? Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
08-01-2022
03:54 AM
Hi, very weird issue here: I have a three-node NiFi 1.16.3 cluster that I deployed several weeks ago in http-mode with all security options disabled for the sake of simplicity since I'm totally new to NiFi. Hence, no login mechanism whatsoever. I've configured several flows and the whole thing has been working flawlessly for weeks. Then I decided to activate all the security options and single-user login. To do so without impacting the operations of the existing cluster, I worked on a new one. Created new keystores and truststores with NiFi tls-toolkit, changed nifi.properties accordingly, created the user on each node with /nifi.sh set-single-user-credentials USER PASSWORD, etc. Once everything was working in the new cluster, I turned the configurations into an Ansible playbook, destroyed the new Nifi cluster and applied the playbook on a new one, created from scratch, several times to make sure the configuration was rock-solid. After all this, I applied the same Ansible playbook to the existing cluster to enable security and single-user login: nifi.security.user.authorizer=single-user-authorizer nifi.security.user.login.identity.provider=single-user-provider Upon restart, the old cluster went up with no issues and all the flows kept working as usual. But I have been able to login to the cluster's web UI only once after the restart of the NiFi service, then for apparently no reason the web interface became accessible only if I enable anonymous mode nifi.security.allow.anonymous.authentication=true Every time I try to login, all I get is a JWT token validation error from org.springframework.security.oauth2.server.resource.InvalidBearerTokenException, even if I am not using Oauth2 at all: 2022-08-01 09:29:31,004 ERROR [NiFi Web Server-511] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: org.springframework.security.oauth2.server.resource.InvalidBearerTokenException: An error occurred while attempting to decode the Jwt: Signed JWT rejected: Another algorithm expected, or no matching key(s) found. Returning Internal Server Error response. org.springframework.security.oauth2.server.resource.InvalidBearerTokenException: An error occurred while attempting to decode the Jwt: Signed JWT rejected: Another algorithm expected, or no matching key(s) found at org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationProvider.getJwt(JwtAuthenticationProvider.java:101) at org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationProvider.authenticate(JwtAuthenticationProvider.java:88) at org.apache.nifi.web.api.AccessResource.getAccessStatus(AccessResource.java:252) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:204) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.apache.nifi.web.filter.ExceptionFilter.doFilter(ExceptionFilter.java:46) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:487) at org.apache.nifi.web.server.filter.DataTransferExcludedDoSFilter.doFilterChain(DataTransferExcludedDoSFilter.java:51) at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:336) at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:301) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.apache.nifi.web.server.log.RequestAuthenticationFilter.doFilterInternal(RequestAuthenticationFilter.java:59) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.apache.nifi.web.security.headers.StrictTransportSecurityFilter.doFilter(StrictTransportSecurityFilter.java:48) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.apache.nifi.web.security.headers.XContentTypeOptionsFilter.doFilter(XContentTypeOptionsFilter.java:48) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.apache.nifi.web.security.headers.XSSProtectionFilter.doFilter(XSSProtectionFilter.java:48) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.apache.nifi.web.security.headers.ContentSecurityPolicyFilter.doFilter(ContentSecurityPolicyFilter.java:47) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.apache.nifi.web.security.headers.XFrameOptionsFilter.doFilter(XFrameOptionsFilter.java:48) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.springframework.security.oauth2.jwt.BadJwtException: An error occurred while attempting to decode the Jwt: Signed JWT rejected: Another algorithm expected, or no matching key(s) found at org.springframework.security.oauth2.jwt.NimbusJwtDecoder.createJwt(NimbusJwtDecoder.java:180) at org.springframework.security.oauth2.jwt.NimbusJwtDecoder.decode(NimbusJwtDecoder.java:137) at org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationProvider.getJwt(JwtAuthenticationProvider.java:97) ... 104 common frames omitted Caused by: com.nimbusds.jose.proc.BadJOSEException: Signed JWT rejected: Another algorithm expected, or no matching key(s) found at com.nimbusds.jwt.proc.DefaultJWTProcessor.process(DefaultJWTProcessor.java:357) at com.nimbusds.jwt.proc.DefaultJWTProcessor.process(DefaultJWTProcessor.java:303) at org.springframework.security.oauth2.jwt.NimbusJwtDecoder.createJwt(NimbusJwtDecoder.java:154) ... 106 common frames omitted I want to stress again that the deployment and configuration of the NiFi cluster is entirely managed by an Ansible playbook that I've tested thoroughly several times on identical clusters created from scratch, and the login procedure has been working flawlessly on them. The issue appears only on an already existing cluster that has been working with the security features disabled for several weeks. Also, I've already deleted cookies and tried to login in incognito mode with both Chrome and Firefox. Could someone please point me in the right direction? This is driving me crazy. Thank you all. W
... View more
Labels:
- Labels:
-
Apache NiFi