Member since
09-12-2023
4
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
305 | 02-22-2024 09:26 PM |
02-22-2024
09:34 PM
Hello all, I am creating a processor group that read from kafka topic and write it to the clickhouse database. I am using stateless mechanism to ensure that when there is a problem during execution, nifi crash, or nifi restarted or clickhouse database return error, kafka offset will not be committed and process will be retry. Unfortunately clickhouse will create a new row for duplicated message. In order to avoid duplicate message, i would like to check first the database and see if i have duplicated message before processing. Have someone create similiar use case as this one?
... View more
Labels:
- Labels:
-
Apache NiFi
02-22-2024
09:26 PM
1 Kudo
I have found a fix for above issue. The trust store password must be assigned with a variable from parameter context, e.g TRUST_STORE_PASSWORD. And when creating a stateless processor, we need to add new sensitive variable TRUST_STORE_PASSWORD. When execute stateless running it will pass the variable to the flow saved in nifi registry.
... View more
02-07-2024
11:10 PM
1 Kudo
Hello support, I am creating another execute stateless from a processor group. This processor group has one consume jms connected with ibm mq. Below is the processor group and consume jms ssl context service. The processor group running fine without execute stateless. After i push the processor group to be referenced with execute stateless processor. After i got following error: Is there a specific way to define the type of truststore file or password for execute stateless.
... View more
Labels:
- Labels:
-
Apache NiFi
12-06-2023
01:28 AM
I am trying to run flow using executestateless by following tutorial here, (190) Kafka Exactly Once with NiFi - YouTube. Nifi version: 1.16.3 running on kubernetes My processor is as follow, And for ProcessXMLStageDB, ConsumeJMS connected to IBM MQ. Jar file and truststore file to connect with IBM MQ is included in the Nifi image, i.e. I create custom image to add the necessary requirement to connect to IBM MQ. This is my ExecuteStateless, This are the details of the executeStateless process, When running i can see following error, {"timestamp": "2023-12-06 08:56:07,478", "level": "INFO", "thread": "Timer-Driven Process Thread-1", "logger": "org.apache.nifi.nar.NarUnpacker", "message": "Expanding 107 NAR files with all processors..."} {"timestamp": "2023-12-06 08:56:07,485", "level": "INFO", "thread": "Timer-Driven Process Thread-1", "logger": "org.apache.nifi.nar.NarUnpacker", "message": "NAR loading process took 7006152 nanoseconds (0 seconds)."} {"timestamp": "2023-12-06 08:56:07,489", "level": "INFO", "thread": "Timer-Driven Process Thread-1", "logger": "o.a.n.s.bootstrap.StatelessBootstrap", "message": "Unpacked NAR files in 12 millis"} {"timestamp": "2023-12-06 08:56:07,508", "level": "ERROR", "thread": "Timer-Driven Process Thread-1", "logger": "o.a.n.p.stateless.ExecuteStateless", "message": "ExecuteStateless[id=3e534fb8-018c-1000-0000-000020557ca5] Processing halted: yielding [1 sec]"} java.lang.NoClassDefFoundError: java/lang/Object at java.base/java.lang.ClassLoader.defineClass1(Native Method) at java.base/java.lang.ClassLoader.defineClass(Unknown Source) at java.base/java.security.SecureClassLoader.defineClass(Unknown Source) at java.base/java.net.URLClassLoader.defineClass(Unknown Source) at java.base/java.net.URLClassLoader$1.run(Unknown Source) at java.base/java.net.URLClassLoader$1.run(Unknown Source) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/java.net.URLClassLoader.findClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Unknown Source) at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(Unknown Source) at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(Unknown Source) at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(Unknown Source) at java.base/java.util.ServiceLoader$2.hasNext(Unknown Source) at java.base/java.util.ServiceLoader$3.hasNext(Unknown Source) at org.apache.nifi.stateless.bootstrap.StatelessBootstrap.getSingleInstance(StatelessBootstrap.java:341) at org.apache.nifi.stateless.bootstrap.StatelessBootstrap.createDataflow(StatelessBootstrap.java:69) at org.apache.nifi.processors.stateless.ExecuteStateless.createDataflow(ExecuteStateless.java:469) at org.apache.nifi.processors.stateless.ExecuteStateless.getDataflow(ExecuteStateless.java:477) at org.apache.nifi.processors.stateless.ExecuteStateless.onTrigger(ExecuteStateless.java:497) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1283) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.lang.ClassNotFoundException: java.lang.Object at java.base/java.net.URLClassLoader.findClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) Anyone has idea what is going wrong ?
... View more
Labels:
- Labels:
-
Apache NiFi