Member since
04-27-2023
2
Posts
0
Kudos Received
0
Solutions
12-19-2023
11:31 PM
Hi All, I got an error while using the ConsumeKafka_2_6 1.19.1 processor . Below is the complete error from the NiFi Bulletin board. 08:49:45 UTCERROR7378316d-4765-1fdd-76cd-cbd9fb7ab699 ConsumeKafka_2_6[id=7378316d-4765-1fdd-76cd-cbd9fb7ab699] Exception while processing data from kafka so will close the lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@3511a438 due to StandardFlowFileRecord[uuid=b1f2cccc-aa9a-4751-9a7c-4ab5eb514555,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1703062164849-3517, container=default, section=445], offset=234005, length=3975],offset=0,name=b1f2cccc-aa9a-4751-9a7c-4ab5eb514555,size=3975] is not known in this session (StandardProcessSession[id=43911398]): org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=b1f2cccc-aa9a-4751-9a7c-4ab5eb514555,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1703062164849-3517, container=default, section=445], offset=234005, length=3975],offset=0,name=b1f2cccc-aa9a-4751-9a7c-4ab5eb514555,size=3975] is not known in this session (StandardProcessSession[id=43911398]) I'm using nifi 1.19.1 along with kafka_3.3.2 version, please let me know if any one knows the root cause of this error.
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache NiFi
04-28-2023
05:20 AM
We have done the NiFi upgradation from 1.18.0 to 1.19.1 and we also upgrade the java version from 8 to 11 and after upgradation we observe nifi using more memory than allocated and caching build in the nifi. we allocate memory to nifi as java.arg.2=-Xms1g and java.arg.3=-Xmx50g. we are using RAM-64 GB and Core-8 and the solid-state high-speed hard disk is 2T. OS is CentOS7.9, ARM64 hardware architecture. we uncomment the Garbage collector from bootstarp.conf but this is not useful for us. we keep the databasr repository, content repository, provenance repository, and flowfile repository in different disks. I need a help to resolve the issue nifi-app.log for reference id=1682607799237-15502, container=contS1R1, section=142], offset=605303, length=7664],offset=0,name=47ffd206-8189-4fbb-9d19-2b82113da956,size=7664], StandardFlowFileRecord[uuid=a06c6ecd-167f-41d6-9ec5-1bcadc37fcf6,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1682607797216-15364, container=contS1R1, section=4], offset=874085, length=11888],offset=0,name=675ad88e-c9e1-4b2f-85a0-c2b842e6c35b,size=11888], StandardFlowFileRecord[uuid=6b21da95-95d9-46f5-8510-4710ef197819,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1682607797216-15364, container=contS1R1, section=4], offset=864517, length=9568],offset=0,name=3b608669-9dee-4286-8865-695ca9f60105,size=9568], StandardFlowFileRecord[uuid=dadb08dd-ee7d-4c70-ade8-14e6b42f89e5,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1682607799314-15505, container=contS1R2, section=145], offset=0, length=12576],offset=0,name=52a965fe-4d18-499f-9f88-3ab1c84f836d,size=12576]] (40.72 KB) from Peer[url=nifi://212.76.88.137:8445] in 2625 milliseconds at a rate of 15.51 KB/sec 2023-04-27 15:03:22,539 INFO [FileSystemRepository Workers Thread-2] o.a.n.c.repository.FileSystemRepository Successfully archived 23 Resource Claims for Container contS1R2 in 2 millis 2023-04-27 15:03:22,663 INFO [FileSystemRepository Workers Thread-2] o.a.n.c.repository.FileSystemRepository Successfully archived 38 Resource Claims for Container contS1R1 in 124 millis 2023-04-27 15:03:22,740 INFO [Timer-Driven Process Thread-3586] o.a.n.c.r.WriteAheadFlowFileRepository Successfully swapped out 10000 FlowFiles from FlowFileQueue[id=ad7834f7-38a1-16f4-4961-0e68a6faca29] to Swap File /data1/flowfile_repository/swap/1682607800425-ad7834f7-38a1-16f4-4961-0e68a6faca29-2ca046bb-321a-43f0-8ffc-12f6eebb77e3.swap 2023-04-27 15:03:22,788 INFO [Timer-Driven Process Thread-2522] o.a.n.r.p.s.SocketFlowFileServerProtocol SocketFlowFileServerProtocol[CommsID=ce5942ca-30bc-44c3-80bd-07ee6391e5cf] Successfully received 22 FlowFiles (455.78 KB) from Peer[url=nifi://212.76.88.137:25294] in 4740 milliseconds at a rate of 96.14 KB/sec 2023-04-27 15:03:22,838 INFO [Timer-Driven Process Thread-1165] o.a.n.r.p.s.SocketFlowFileServerProtocol SocketFlowFileServerProtocol[CommsID=eac02335-3ac7-42bd-8779-4253e20fd36f] Successfully received 25 FlowFiles (168.09 KB) from Peer[url=nifi://212.76.88.137:9047] in 1217 milliseconds at a rate of 138.06 KB/sec
... View more
Labels: