Member since
03-22-2019
28
Posts
1
Kudos Received
0
Solutions
01-10-2022
09:03 AM
Hi , Nifi is not able to form a cluster after restart , this was working fine for months without any issues, only after restart i am seeing issues , can anyone please help ? i have below zookeeper configuration and nifi.properties and nifi-app.log information. we are using external zookeeper. zoo.cfg initLimit=100
autopurge.purgeInterval=24
autopurge.snapRetainCount=30
syncLimit=50
tickTime=2000
dataDir=/ngs/app/xxx/zookeeper-data-356_1
admin.enableServer=true
admin.serverPort=9990
standaloneEnabled=false
server.1=node1:2887:3887;2184
server.2=node2:2886:3886;2182
server.3=node3:2889:3889;2183
quorum.cnxn.threads.size=20
4lw.commands.whitelist=mntr,stat nifi.properties nifi.state.management.configuration.file=./conf/state-management.xml
# The ID of the local state provider
nifi.state.management.provider.local=local-provider
nifi.state.management.provider.cluster=zk-provider
nifi.state.management.embedded.zookeeper.start=false
nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties
nifi.cluster.is.node=true
nifi.cluster.node.address=node-xxxx
nifi.cluster.node.protocol.port=11443
nifi.cluster.node.protocol.threads=10
nifi.cluster.node.protocol.max.threads=50
nifi.cluster.node.event.history.size=25
nifi.cluster.node.connection.timeout=10 sec
nifi.cluster.node.read.timeout=10 sec
nifi.cluster.node.max.concurrent.requests=100
nifi.cluster.firewall.file=
nifi.cluster.flow.election.max.wait.time=1 mins
nifi.cluster.flow.election.max.candidates=
nifi.zookeeper.connect.string=xxxx:2182,xxxx:2183,xxxx:2184
nifi.zookeeper.connect.timeout=3 secs
nifi.zookeeper.session.timeout=3 secs
nifi.zookeeper.root.node=/nifi Nifi-app.log on node 1: 2022-01-09 05:55:56,151 INFO [main] o.a.n.c.repository.FileSystemRepository Initializing FileSystemRepository with 'Always Sync' set to false
2022-01-09 05:55:56,379 INFO [main] o.apache.nifi.controller.FlowController Checking if there is already a Cluster Coordinator Elected...
2022-01-09 05:55:56,521 INFO [main] org.apache.curator.utils.Compatibility Using emulated InjectSessionExpiration
2022-01-09 05:55:56,590 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting
2022-01-09 05:55:56,609 INFO [main] org.apache.zookeeper.common.X509Util Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2022-01-09 05:55:56,622 INFO [main] org.apache.zookeeper.ClientCnxnSocket jute.maxbuffer value is 4194304 Bytes
2022-01-09 05:55:56,668 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Default schema
2022-01-09 05:55:56,719 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED
2022-01-09 05:55:56,757 INFO [main-EventThread] o.a.c.framework.imps.EnsembleTracker New config event received: {server.2=xxxxxx:2886:3886:participant;0.0.0.0:2182, server.1=xxxxxx:2887:3887:participant;0.0.0.0:2184, server.3=xxxxxx:2889:3889:participant;0.0.0.0:2183, version=0}
2022-01-09 05:55:56,777 INFO [main-EventThread] o.a.c.framework.imps.EnsembleTracker New config event received: {server.2=xxxxxx:2886:3886:participant;0.0.0.0:2182, server.1=xxxxxx:2887:3887:participant;0.0.0.0:2184, server.3=xxxxxx:2889:3889:participant;0.0.0.0:2183, version=0}
2022-01-09 05:55:56,790 INFO [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
2022-01-09 05:55:56,913 INFO [main] o.apache.nifi.controller.FlowController It appears that no Cluster Coordinator has been Elected yet. Registering for Cluster Coordinator Role.
2022-01-09 05:55:56,914 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=true] Registered new Leader Selector for role Cluster Coordinator; this node is an active participant in the election.
2022-01-09 05:55:56,917 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting
2022-01-09 05:55:56,918 INFO [main] org.apache.zookeeper.ClientCnxnSocket jute.maxbuffer value is 4194304 Bytes
2022-01-09 05:55:56,920 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Default schema
2022-01-09 05:55:56,924 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Registered new Leader Selector for role Cluster Coordinator; this node is an active participant in the election.
2022-01-09 05:55:56,924 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] started
2022-01-09 05:55:56,924 INFO [main] o.a.n.c.c.h.AbstractHeartbeatMonitor Heartbeat Monitor started
2022-01-09 05:55:56,952 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED
.....
2022-01-09 05:56:15,810 INFO [main] o.apache.nifi.controller.FlowController Successfully synchronized controller with proposed flow
2022-01-09 05:56:18,732 INFO [main] o.a.nifi.controller.StandardFlowService Connecting Node: rn-ssenpcit-lapp205.rno.apple.com:8086
2022-01-09 05:56:18,771 WARN [main] o.a.nifi.controller.StandardFlowService There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Cluster Coordinator and will attempt to connect to cluster again
2022-01-09 05:56:18,771 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Attempted to register Leader Election for role 'Cluster Coordinator' but this role is already registered
2022-01-09 05:56:19,787 WARN [main] o.a.nifi.controller.StandardFlowService There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Cluster Coordinator and will attempt to connect to cluster again
2022-01-09 05:56:19,787 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Attempted to register Leader Election for role 'Cluster Coordinator' but this role is already registered
2022-01-09 05:56:20,798 WARN [main] o.a.nifi.controller.StandardFlowService There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Cluster Coordinator and will attempt to connect to cluster again Nifi-app.log on node 2 : 2022-01-09 05:55:40,827 INFO [main] o.apache.nifi.controller.FlowController Checking if there is already a Cluster Coordinator Elected...
2022-01-09 05:55:40,893 INFO [main] org.apache.curator.utils.Compatibility Using emulated InjectSessionExpiration
2022-01-09 05:55:40,938 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting
2022-01-09 05:55:40,950 INFO [main] org.apache.zookeeper.common.X509Util Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2022-01-09 05:55:40,957 INFO [main] org.apache.zookeeper.ClientCnxnSocket jute.maxbuffer value is 4194304 Bytes
2022-01-09 05:55:40,985 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Default schema
2022-01-09 05:55:41,031 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED
2022-01-09 05:55:41,057 INFO [main-EventThread] o.a.c.framework.imps.EnsembleTracker New config event received: {server.2=rn-ssenpcit-lapp206.rno.apple.com:2886:3886:participant;0.0.0.0:2182, server.1=rn-ssenpcit-lapp206.rno.apple.com:2887:3887:participant;0.0.0.0:2184, server.3=rn-ssenpcit-lapp206.rno.apple.com:2889:3889:participant;0.0.0.0:2183, version=0}
2022-01-09 05:55:41,067 INFO [main-EventThread] o.a.c.framework.imps.EnsembleTracker New config event received: {server.2=rn-ssenpcit-lapp206.rno.apple.com:2886:3886:participant;0.0.0.0:2182, server.1=rn-ssenpcit-lapp206.rno.apple.com:2887:3887:participant;0.0.0.0:2184, server.3=rn-ssenpcit-lapp206.rno.apple.com:2889:3889:participant;0.0.0.0:2183, version=0}
2022-01-09 05:55:41,069 INFO [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
2022-01-09 05:55:41,186 INFO [main] o.apache.nifi.controller.FlowController It appears that no Cluster Coordinator has been Elected yet. Registering for Cluster Coordinator Role.
2022-01-09 05:55:41,187 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=true] Registered new Leader Selector for role Cluster Coordinator; this node is an active participant in the election.
2022-01-09 05:55:41,191 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting
2022-01-09 05:55:41,192 INFO [main] org.apache.zookeeper.ClientCnxnSocket jute.maxbuffer value is 4194304 Bytes
2022-01-09 05:55:41,195 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Default schema
2022-01-09 05:55:41,201 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Registered new Leader Selector for role Cluster Coordinator; this node is an active participant in the election.
2022-01-09 05:55:41,202 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] started
2022-01-09 05:55:41,202 INFO [main] o.a.n.c.c.h.AbstractHeartbeatMonitor Heartbeat Monitor started
2022-01-09 05:55:41,205 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED
......
2022-01-09 05:55:57,209 INFO [main] o.apache.nifi.controller.FlowController Successfully synchronized controller with proposed flow
2022-01-09 05:56:00,250 INFO [main] o.a.nifi.controller.StandardFlowService Connecting Node: rn-ssenpcit-lapp206.rno.apple.com:8086
2022-01-09 05:56:00,251 WARN [main] o.a.nifi.controller.StandardFlowService There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Cluster Coordinator and will attempt to connect to cluster again
2022-01-09 05:56:00,251 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Attempted to register Leader Election for role 'Cluster Coordinator' but this role is already registered
2022-01-09 05:56:01,252 WARN [main] o.a.nifi.controller.StandardFlowService There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Cluster Coordinator and will attempt to connect to cluster again
2022-01-09 05:56:01,252 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Attempted to register Leader Election for role 'Cluster Coordinator' but this role is already registered
2022-01-09
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Zookeeper
09-23-2021
10:06 AM
Hi, We have recently made our external ZooKeeper's implement SSL, Now Nifi is failing to startup. 2021-09-23 15:46:01,711 ERROR [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl Background retry gave up
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss
at org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:972)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:943)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:66)
at org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:346)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834) Nifi version we are using is 1.10.0 using TLS properties via nifi.properties as below nifi.zookeeper.client.secure=true
nifi.zookeeper.security.keystore=./conf/keystore
nifi.zookeeper.security.keystoreType=jks
nifi.zookeeper.security.keystorePasswd=password
nifi.zookeeper.security.truststore=./conf/truststore
nifi.zookeeper.security.truststoreType=jks
nifi.zookeeper.security.truststorePasswd=password
nifi.zookeeper.connect.string=host1:2185,host2:2186,host3:2187
nifi.zookeeper.connect.timeout=10 secs
nifi.zookeeper.session.timeout=10 secs
nifi.zookeeper.root.node=/nifi However i see that TLS support for nifi.properties is available only in 1.13.0 ? as per this jira https://issues.apache.org/jira/browse/NIFI-7357 can you please let me know how to use TLS properties via nifi.properties in Nifi version : 1.10.0
... View more
Labels:
- Labels:
-
Apache NiFi
-
Security
05-18-2020
04:32 PM
How to connect to NIFI GetMongo using x509 connection with certificates ?
... View more
- Tags:
- NiFi
Labels:
- Labels:
-
Apache NiFi
11-25-2019
10:13 AM
It is working fine now with 3.5.6 version of zookeeper
... View more
11-18-2019
11:32 AM
Thanks @Shelton , will give it a try and update here later.
... View more
11-17-2019
02:48 PM
Upgrading to Java 11 is a mandate across our ORG. I was able to compile ( On MAC with Java 11) and seems to be working fine. however see above issue when installing Nifi-1.10.0 with external zookeeper.
... View more
11-17-2019
11:19 AM
Can anyone please help ?
... View more
11-15-2019
08:48 AM
Hi,
I am having issues with the zookeeper (zookeeper-3.4.13) with latest nifi version of Nifi-1.10.0 , below is the stack trace. I am upgrading from nifi-1.9.0 . let me know if latest version of Nifi needs different version of zookeeper ? or the issue is somewhere else.
Not : With Java 8 and nifi-1.9.0 everything works
Actual Logs using Java 11
2019-11-15 03:40:09,202 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED 2019-11-15 03:40:09,241 INFO [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting 2019-11-15 03:40:09,355 INFO [main] o.apache.nifi.controller.FlowController It appears that no Cluster Coordinator has been Elected yet. Registering for Cluster Coordinator Role. 2019-11-15 03:40:09,356 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=true] Registered new Leader Selector for role Cluster Coordinator; this node is an active participant in the election. 2019-11-15 03:40:09,360 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting 2019-11-15 03:40:09,361 INFO [main] org.apache.zookeeper.ClientCnxnSocket jute.maxbuffer value is 4194304 Bytes 2019-11-15 03:40:09,365 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Default schema 2019-11-15 03:40:09,371 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Registered new Leader Selector for role Cluster Coordinator; this node is an active participant in the election. 2019-11-15 03:40:09,372 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] started 2019-11-15 03:40:09,372 INFO [main] o.a.n.c.c.h.AbstractHeartbeatMonitor Heartbeat Monitor started 2019-11-15 03:40:09,375 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED 2019-11-15 03:40:09,387 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@43eff6be Connection State changed to CONNECTED 2019-11-15 03:40:09,540 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: SUSPENDED 2019-11-15 03:40:09,540 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@43eff6be Connection State changed to SUSPENDED 2019-11-15 03:40:10,490 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: RECONNECTED 2019-11-15 03:40:10,490 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@43eff6be Connection State changed to RECONNECTED 2019-11-15 03:40:10,634 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: SUSPENDED 2019-11-15 03:40:10,636 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@43eff6be Connection State changed to SUSPENDED 2019-11-15 03:40:10,835 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: RECONNECTED 2019-11-15 03:40:10,841 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@43eff6be Connection State changed to RECONNECTED 2019-11-15 03:40:10,952 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: SUSPENDED 2019-11-15 03:40:10,959 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@43eff6be Connection State changed to SUSPENDED 2019-11-15 03:40:11,017 ERROR [main-EventThread] o.a.c.f.imps.CuratorFrameworkImpl Background operation retry gave up org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) at org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:862) at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:647) at org.apache.curator.framework.imps.WatcherRemovalFacade.processBackgroundOperation(WatcherRemovalFacade.java:152) at org.apache.curator.framework.imps.FindAndDeleteProtectedNodeInBackground$2.processResult(FindAndDeleteProtectedNodeInBackground.java:104) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:630) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2019-11-15 03:40:11,450 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: RECONNECTED 2019-11-15 03:40:11,455 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@43eff6be Connection State changed to RECONNECTED 2019-11-15 03:40:11,551 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: SUSPENDED 2019-11-15 03:40:11,552 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@43eff6be Connection State changed to SUSPENDED 2019-11-15 03:40:11,552 ERROR [main-EventThread] o.a.c.f.imps.CuratorFrameworkImpl Background operation retry gave up
Comparision of Java 8 with Nifi-1.9.0 and Java11 Nifi-1.10.0 logs below
===================== JAVA 8 working ================================================ 2019-11-15 18:07:44,302 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@196ca821 finished recovering records. Performing Checkpoint to ensure proper state of Partitions before updates 2019-11-15 18:07:44,303 INFO [main] org.wali.MinimalLockingWriteAheadLog Successfully recovered 1800 records in 18 milliseconds 2019-11-15 18:07:44,330 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@196ca821 checkpointed with 1800 Records and 0 Swap Files in 26 milliseconds (Stop-the-world time = 4 milliseconds, Clear Edit Logs time = 3 millis), max Transaction ID 137772318 2019-11-15 18:07:45,253 INFO [main] o.a.n.c.c.node.NodeClusterCoordinator Resetting cluster node statuses from {} to {rn-ssenpcit-lapp205.rno.apple.com:8086=NodeConnectionStatus[nodeId=rn-ssenpcit-lapp205.rno.apple.com:8086, state=DISCONNECTED, Disconnect Code=Has Not Yet Connected to Cluster, Disconnect Reason=Has Not Yet Connected to Cluster, updateId=1], rn-ssenpcit-lapp206.rno.apple.com:8086=NodeConnectionStatus[nodeId=rn-ssenpcit-lapp206.rno.apple.com:8086, state=DISCONNECTED, Disconnect Code=Has Not Yet Connected to Cluster, Disconnect Reason=Has Not Yet Connected to Cluster, updateId=0]} 2019-11-15 18:07:45,363 INFO [main] o.a.n.r.v.FileBasedVariableRegistry Loaded 97 properties from system properties and environment variables 2019-11-15 18:07:45,364 INFO [main] o.a.n.r.v.FileBasedVariableRegistry Loaded a total of 97 properties. Including precedence overrides effective accessible registry key size is 97 2019-11-15 18:07:45,520 INFO [main] o.a.n.c.r.WriteAheadFlowFileRepository Initialized FlowFile Repository using 256 partitions 2019-11-15 18:07:46,240 INFO [main] o.a.n.p.store.WriteAheadStorePartition After recovering ./provenance_repository, next Event ID to be generated will be 148085136 2019-11-15 18:07:47,399 INFO [pool-23-thread-1] o.a.n.p.index.lucene.LuceneEventIndex Determined that Max Event ID indexed for Partition default is approximately 148084987 based on index ./provenance_repository/index-1573685499416 2019-11-15 18:07:47,403 INFO [pool-23-thread-1] o.a.n.p.store.WriteAheadStorePartition The last Provenance Event indexed for partition default is 148074987, but the last event written to partition has ID 148085135. Re-indexing up to the last 10148 events to ensure that the Event Index is accurate and up-to-date 2019-11-15 18:07:49,963 INFO [pool-23-thread-1] o.a.n.p.store.WriteAheadStorePartition Finished re-indexing 10149 events across 1 files for ./provenance_repository in 2.559 seconds 2019-11-15 18:07:49,973 INFO [main] o.a.n.c.repository.FileSystemRepository Maximum Threshold for Container default set to 35101377658 bytes; if volume exceeds this size, archived data will be deleted until it no longer exceeds this size 2019-11-15 18:07:49,975 INFO [main] o.a.n.c.repository.FileSystemRepository Initializing FileSystemRepository with 'Always Sync' set to false 2019-11-15 18:07:50,935 INFO [main] o.apache.nifi.controller.FlowController Checking if there is already a Cluster Coordinator Elected... 2019-11-15 18:07:51,015 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting 2019-11-15 18:07:51,340 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED 2019-11-15 18:07:51,362 INFO [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting 2019-11-15 18:07:51,370 INFO [main] o.apache.nifi.controller.FlowController The Election for Cluster Coordinator has already begun (Leader is rn-ssenpcit-lapp206.rno.apple.com:11443). Will not register to be elected for this role until after connecting to the cluster and inheriting the cluster's flow. 2019-11-15 18:07:51,372 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=true] Registered new Leader Selector for role Cluster Coordinator; this node is a silent observer in the election. 2019-11-15 18:07:51,373 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting 2019-11-15 18:07:51,376 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Registered new Leader Selector for role Cluster Coordinator; this node is a silent observer in the election. 2019-11-15 18:07:51,379 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] started 2019-11-15 18:07:51,379 INFO [main] o.a.n.c.c.h.AbstractHeartbeatMonitor Heartbeat Monitor started 2019-11-15 18:07:51,385 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED 2019-11-15 18:07:55,430 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@6ca9f1d0{nifi-api,/nifi-api,file:///ngs/app/ssewiret/nifi/work/jetty/nifi-web-api-1.9.0.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.9.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.9.0.war} 2019-11-15 18:07:55,831 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=135ms 2019-11-15 18:07:55,862 INFO [main] o.e.j.s.h.C._nifi_content_viewer No Spring WebApplicationInitializer types detected on classpath 2019-11-15 18:07:56,194 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@4a467f08{nifi-content-viewer,/nifi-content-viewer,file:///ngs/app/ssewiret/nifi/work/jetty/nifi-web-content-viewer-1.9.0.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.9.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-content-viewer-1.9.0.war} 2019-11-15 18:07:56,218 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=9ms 2019-11-15 18:07:56,219 WARN [main] o.e.j.webapp.StandardDescriptorProcessor Duplicate mapping from / to default ======================== JAVA 11 NOT WORKING ============================================ 2019-11-15 20:02:32,893 ERROR [main] org.apache.nifi.encrypt.StringEncryptor ******************************************************************************** 2019-11-15 20:02:32,996 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@56c856c0 finished recovering records. Performing Checkpoint to ensure proper state of Partitions before updates 2019-11-15 20:02:32,996 INFO [main] org.wali.MinimalLockingWriteAheadLog Successfully recovered 1800 records in 31 milliseconds 2019-11-15 20:02:33,015 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@56c856c0 checkpointed with 1800 Records and 0 Swap Files in 19 milliseconds (Stop-the-world time = 3 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID 137772323 2019-11-15 20:02:33,517 INFO [main] o.a.n.c.c.node.NodeClusterCoordinator Resetting cluster node statuses from {} to {rn-ssenpcit-lapp205.rno.apple.com:8086=NodeConnectionStatus[nodeId=rn-ssenpcit-lapp205.rno.apple.com:8086, state=DISCONNECTED, Disconnect Code=Has Not Yet Connected to Cluster, Disconnect Reason=Has Not Yet Connected to Cluster, updateId=1], rn-ssenpcit-lapp206.rno.apple.com:8086=NodeConnectionStatus[nodeId=rn-ssenpcit-lapp206.rno.apple.com:8086, state=DISCONNECTED, Disconnect Code=Has Not Yet Connected to Cluster, Disconnect Reason=Has Not Yet Connected to Cluster, updateId=0]} 2019-11-15 20:02:33,569 INFO [main] o.a.n.r.v.FileBasedVariableRegistry Loaded 95 properties from system properties and environment variables 2019-11-15 20:02:33,569 INFO [main] o.a.n.r.v.FileBasedVariableRegistry Loaded a total of 95 properties. Including precedence overrides effective accessible registry key size is 95 2019-11-15 20:02:33,629 INFO [main] o.a.n.c.r.WriteAheadFlowFileRepository Initialized FlowFile Repository using 256 partitions 2019-11-15 20:02:33,768 INFO [main] o.a.n.p.store.WriteAheadStorePartition After recovering ./provenance_repository, next Event ID to be generated will be 148085136 2019-11-15 20:02:33,835 WARN [main] o.a.n.p.index.lucene.LuceneEventIndex Lucene Index ./provenance_repository/index-1573685499416 could not be opened. Assuming that index is defunct and will re-index events belonging to this index. 2019-11-15 20:02:33,835 INFO [main] o.a.n.p.index.lucene.LuceneEventIndex Determined that Lucene Index Directory ./provenance_repository/index-1573685499416 is defunct. Will destroy and rebuild index 2019-11-15 20:02:33,839 INFO [main] o.a.n.p.index.lucene.LuceneEventIndex Will avoid re-indexing Provenance Events because the newest index is defunct, so it will be re-indexed in the background 2019-11-15 20:02:33,839 INFO [main] o.a.n.p.WriteAheadProvenanceRepository Provenance Event Index indicates that no events should be re-indexed upon startup. Will not wait for re-indexing to occur. 2019-11-15 20:02:33,840 INFO [main] o.a.n.c.repository.FileSystemRepository Maximum Threshold for Container default set to 35101377658 bytes; if volume exceeds this size, archived data will be deleted until it no longer exceeds this size 2019-11-15 20:02:33,842 INFO [main] o.a.n.c.repository.FileSystemRepository Initializing FileSystemRepository with 'Always Sync' set to false 2019-11-15 20:02:34,199 INFO [main] o.apache.nifi.controller.FlowController Checking if there is already a Cluster Coordinator Elected... 2019-11-15 20:02:34,328 INFO [main] org.apache.curator.utils.Compatibility Using emulated InjectSessionExpiration 2019-11-15 20:02:34,456 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting 2019-11-15 20:02:34,505 INFO [main] org.apache.zookeeper.common.X509Util Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation 2019-11-15 20:02:34,547 INFO [main] org.apache.zookeeper.ClientCnxnSocket jute.maxbuffer value is 4194304 Bytes 2019-11-15 20:02:34,743 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Default schema 2019-11-15 20:02:34,983 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED 2019-11-15 20:02:35,083 INFO [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting 2019-11-15 20:02:35,219 INFO [main] o.apache.nifi.controller.FlowController It appears that no Cluster Coordinator has been Elected yet. Registering for Cluster Coordinator Role. 2019-11-15 20:02:35,220 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=true] Registered new Leader Selector for role Cluster Coordinator; this node is an active participant in the election. 2019-11-15 20:02:35,223 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting 2019-11-15 20:02:35,236 INFO [main] org.apache.zookeeper.ClientCnxnSocket jute.maxbuffer value is 4194304 Bytes 2019-11-15 20:02:35,250 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Default schema 2019-11-15 20:02:35,257 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] Registered new Leader Selector for role Cluster Coordinator; this node is an active participant in the election. 2019-11-15 20:02:35,257 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=false] started 2019-11-15 20:02:35,257 INFO [main] o.a.n.c.c.h.AbstractHeartbeatMonitor Heartbeat Monitor started 2019-11-15 20:02:35,283 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED 2019-11-15 20:02:35,290 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@4d490d2c Connection State changed to CONNECTED 2019-11-15 20:02:35,565 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: SUSPENDED 2019-11-15 20:02:35,568 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@4d490d2c Connection State changed to SUSPENDED 2019-11-15 20:02:36,567 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: RECONNECTED 2019-11-15 20:02:36,567 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@4d490d2c Connection State changed to RECONNECTED 2019-11-15 20:02:36,680 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: SUSPENDED 2019-11-15 20:02:36,691 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@4d490d2c Connection State changed to SUSPENDED 2019-11-15 20:02:36,729 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: RECONNECTED 2019-11-15 20:02:36,730 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorLeaderElectionManager org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager$ElectionListener@4d490d2c Connection State changed to RECONNECTED 2019-11-15 20:02:36,859 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: SUSPENDED
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Zookeeper
11-14-2019
09:16 PM
1 Kudo
Yes , it was picking up openJDK 9 which was in the classpath, i pointed to right JDK 11 and it is working now.
... View more
11-14-2019
01:38 PM
I have installed latest version of Nifi-1.10.0 , everything works locally , however on my Linux server i am having issues when i start NIFI , below is the stack trace.
2019-11-14 21:22:31,464 INFO [main] org.apache.nifi.NiFi Launching NiFi... 2019-11-14 21:22:31,812 INFO [main] org.apache.nifi.security.kms.CryptoUtils Determined default nifi.properties path to be '/ngs/app/ssewiret/nifi/./conf/nifi.properties' 2019-11-14 21:22:31,818 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 143 properties from /ngs/app/ssewiret/nifi/./conf/nifi.properties 2019-11-14 21:22:31,826 INFO [main] org.apache.nifi.NiFi Loaded 143 properties 2019-11-14 21:22:31,838 INFO [main] org.apache.nifi.BootstrapListener Started Bootstrap Listener, Listening for incoming requests on port 33317 2019-11-14 21:22:31,862 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap 2019-11-14 21:22:31,909 INFO [main] org.apache.nifi.nar.NarUnpacker Expanding 198 NAR files with all processors... 2019-11-14 21:22:31,933 WARN [main] org.apache.nifi.nar.NarUnpacker Unable to load NAR library bundles due to java.util.zip.ZipException: zip END header not found Will proceed without loading any further Nar bundles 2019-11-14 21:22:31,942 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.lang.IllegalStateException: Unable to locate Jetty bundle. java.lang.IllegalStateException: Unable to locate Jetty bundle. at org.apache.nifi.nar.NarClassLoaders.load(NarClassLoaders.java:211) at org.apache.nifi.nar.NarClassLoaders.init(NarClassLoaders.java:119) at org.apache.nifi.NiFi.<init>(NiFi.java:134) at org.apache.nifi.NiFi.<init>(NiFi.java:72) at org.apache.nifi.NiFi.main(NiFi.java:301) 2019-11-14 21:22:31,944 INFO [Thread-0] org.apache.nifi.NiFi Initiating shutdown of Jetty web server... 2019-11-14 21:22:31,946 INFO [Thread-0] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
Thanks for the help.
... View more
Labels:
- Labels:
-
Apache NiFi
07-11-2019
07:48 PM
I am still facing the same issue any help would be appreciated.
... View more
07-01-2019
05:55 PM
@Jay Kumar SenSharma Below are the details : ~ git:(master) ✗ grep 'nifi-web-ui' /Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/logs/nifi-app.log 2019-06-30 21:41:27,762 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: /Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./work/nar/framework/nifi-framework-nar-1.9.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-ui-1.9.0.war with context path set to /nifi 2019-06-30 21:41:42,074 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@509e4bc4{nifi,/nifi,file:///Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/work/jetty/nifi-web-ui-1.9.0.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.9.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-ui-1.9.0.war} ~ git:(master) ✗ ls -lart /Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/work/nar/framework/nifi-framework-nar-1.9.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.9.0.war -rw-r--r-- 1 samaraarkotti staff 950040 Feb 21 17:48 /Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/work/nar/framework/nifi-framework-nar-1.9.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.9.0.war Attached are the logs: nifi-app.txt nifi-bootstrap.txt nifi-user.txt
... View more
06-29-2019
02:56 AM
can anyone really help ?
... View more
06-28-2019
04:01 PM
@Jim Streuli were you able to resolve this issue? , i have upgraded my Mac OS to Catalina and see the same issue. getting HTTP ERROR 404.
... View more
06-28-2019
05:52 AM
re-posting.. didn't added it as comment. Below are the CURL Commands output. CurlCommands.txt ➜ ~ git:(master) ✗ netstat -an | grep 8080 tcp4 0 0 *.8080 *.* LISTEN ➜ ~ git:(master) ✗ ps -ef | grep nifi 501 19949 1 0 9:57PM ttys001 0:00.00 /bin/sh ./nifi.sh start 501 19951 19949 0 9:57PM ttys001 0:01.29 /Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/bin/java -cp /Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/conf:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/lib/bootstrap/* -Xms12m -Xmx24m -Dorg.apache.nifi.bootstrap.config.log.dir=/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/logs -Dorg.apache.nifi.bootstrap.config.pid.dir=/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/run -Dorg.apache.nifi.bootstrap.config.file=/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/conf/bootstrap.conf org.apache.nifi.bootstrap.RunNiFi start 501 19952 19951 0 9:57PM ttys001 2:18.73 /Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/bin/java -classpath /Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./conf:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/nifi-properties-1.9.0.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/jetty-schemas-3.1.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/slf4j-api-1.7.25.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/nifi-framework-api-1.9.0.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/javax.servlet-api-3.1.0.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/jcl-over-slf4j-1.7.25.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/logback-classic-1.2.3.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/jul-to-slf4j-1.7.25.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/nifi-runtime-1.9.0.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/log4j-over-slf4j-1.7.25.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/logback-core-1.2.3.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/nifi-nar-utils-1.9.0.jar:/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./lib/nifi-api-1.9.0.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx2048M -Xms2048M -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000 -Djavax.security.auth.useSubjectCredsOnly=true -Djava.security.egd=file:/dev/urandom -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+UseG1GC -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dnifi.properties.file.path=/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/./conf/nifi.properties -Dnifi.bootstrap.listen.port=58309 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=/Users/samaraarkotti/Documents/example-nifi-1.9.0-deploy/nifi-1.9.0/logs org.apache.nifi.NiFi 501 19975 1382 0 9:57PM ttys001 0:00.03 tail -f ../logs/nifi-app.log 501 20666 14587 0 10:06PM ttys002 0:00.00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn nifi
... View more
06-28-2019
04:10 AM
I have downloaded Nifi on my laptop running Mac OS Catalina Public Beta 2 and not able to access the UI. It is up and running and able to access all the REST API's shown below , http://127.0.0.1:8080/nifi-api/flow/processor-types http://127.0.0.1:8080/nifi-api/flow/controller-service-types http://127.0.0.1:8080/nifi-api/flow/reporting-task-types http://127.0.0.1:8080/nifi-api/controller/config http://127.0.0.1:8080/nifi-api/resources http://127.0.0.1:8080/nifi-api/system-diagnostics However UI is not accessible , getting below error. Note this is very specific to Mac OS Catalina public beta. it is working fine in Mojave . Nothing stands out in any of the logs. HTTP ERROR 404 Problem accessing /nifi/. Reason: Not Found Powered by Jetty:// 9.4.11.v20180605 For testing purpose , i have downloaded Nifi-Registry and it is also having the same issue.
... View more
Labels:
- Labels:
-
Apache NiFi
04-10-2019
06:41 PM
Thanks @Matt Clarke i will set the configure concurrent task for this processor to be 1 , my rest of the processor's in the flow are set to 2 concurrent task's . which should be fine i guess.
... View more
04-10-2019
05:10 PM
I am seeing below messages and it is messing up the SRC_OFFSET in this case, i have configured UpdateAttribute with 2 concurrent Tasks. mine is a 3 node cluster, Nifi version 1.9.0
... View more
Labels:
- Labels:
-
Apache NiFi
04-01-2019
04:07 PM
Thanks @kdoran currently i am doing POC in development environment itself, and Spnego is not working, tried various methods to make it work. deleting the authorizations.xml , users.xml, flowfiles archive ...etc and restarted everything again. but spnego is not working. when i launch nifi it is alway's showing login page. tried with all browsers. am i missing something here ? posted the same question seperately : https://community.hortonworks.com/questions/243723/kerberos-spnego-authentication.html
... View more
03-30-2019
02:48 AM
My SPNEGO is not working , my settings are exactly like @Scott Aslan what i have observed is , if you login using username/password for first time , and if you clear browser cache launch nifi it will not show login page until you do kinit again , once you do kinit it will present the login screen again next time you launch nifi , I have always launch from terminal after seeing this post @kdoran. how can i avoid login page using spnego ? using firefox, and nifi version 1.9.0
... View more
