Member since
12-03-2017
157
Posts
26
Kudos Received
11
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2746 | 11-03-2023 12:17 AM | |
| 5072 | 12-12-2022 09:16 PM | |
| 2076 | 07-14-2022 03:25 AM | |
| 2880 | 07-28-2021 04:42 AM | |
| 4070 | 06-23-2020 10:08 PM |
07-14-2022
03:25 AM
testRunner.setAllowSynchronousSessionCommits(true); Helped me to get rid of this error
... View more
07-13-2022
01:14 AM
Hello Experts, I am using org.apache.nifi.util.TestRunners in Nifi custom processor unit test cases, and code something like below.. testRunner = TestRunners.newTestRunner(O9QueueRouting.class); . . . . MockFlowFile ff = this.testRunner.enqueue(new ByteArrayInputStream(ffContents.getBytes(StandardCharsets.UTF_8))); Map<String, String> attrs = new HashMap<String, String>(); attrs.put("routingheader", "headervalue"); attrs.put("kafka.topic", "topic1"); ff.putAttributes(attrs); testRunner.run(); It was working earlier now started giving error like below - [ERROR] O9QueueRoutingTest.test:73 java.lang.RuntimeException: As of version 1.14.0, ProcessSession.commit() should be avoided when possible. See JavaDocs for explanations. Instead, use commitAsync(), commitAsync(Runnable), or commitAsync(Runnable, Consumer<Throwable>). However, if this is not possible, ProcessSession.commit() may still be used, but this must be explicitly enabled by calling TestRunner. Any idea how this can be resolved ? How to explicitly enable by calling TestRunner ? Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
07-09-2022
05:24 AM
It depends on your use case - how many tables, how much records each table has, one time migration vs cdc etc. For one time migration (for till date) , if data is huge then its better to run sql paginated query instead of full table select so that resulting flow file will not be very huge and hence no impact on nifi performance. If you have multiple tables then may be you can add a generate flow file processor for triggering the flow with all the tables names in json and split the json and send each table name to ExecuteSQLQuery processor to execute and get data.. If CDC need migrated then may be you can schedule the flow very frequently and query the data based on datetime range if possible or any incremental values (which can be cached using nifi cache for new run)
... View more
07-05-2022
10:50 PM
1 Kudo
Issue created : https://issues.apache.org/jira/browse/NIFI-10197
... View more
07-01-2022
11:39 AM
Hello Experts, I am trying to upgrade nifi from 1.12.1 version to 1.16.3. I have a script 'ECMAScript' which was working fine in 1.12.1and now behaving different in 1.16.3. Below is the portion of code giving error in 1.16 version when 'xyzId' is null. But it just logs null as expected in 1.12 version var xyzId = flowFile.getAttribute('xyzId'); log.info(xyzId); Below is the error in 1.16 version when null value received for above line (log.info() line throws below error) imer-Driven Process Thread-8] o.a.nifi.processors.script.ExecuteScript ExecuteScript[id=bae38131-0181-1000-29af-196312cdd9d9] ExecuteScript[id=bae38131-0181-1000-29af-196312cdd9d9] failed to process due to org.apache.nifi.processor.exception.ProcessException: java.lang.RuntimeException: java.lang.NoSuchMethodException: Can't unambiguously select between fixed arity signatures [(org.apache.nifi.logging.LogMessage), (java.lang.String)] of the method org.apache.nifi.controller.TerminationAwareLogger.info for argument types [null]; rolling back session org.apache.nifi.processor.exception.ProcessException: java.lang.RuntimeException: java.lang.NoSuchMethodException: Can't unambiguously select between fixed arity signatures [(org.apache.nifi.logging.LogMessage), (java.lang.String)] of the method org.apache.nifi.controller.TerminationAwareLogger.info for argument types [null] at org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:245) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1283) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodException: Can't unambiguously select between fixed arity signatures [(org.apache.nifi.logging.LogMessage), (java.lang.String)] of the method org.apache.nifi.controller.TerminationAwareLogger.info for argument types [null] at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:397) at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:449) at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:406) at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:402) at jdk.nashorn.api.scripting.NashornScriptEngine.eval(NashornScriptEngine.java:155) at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264) at org.apache.nifi.script.impl.JavascriptScriptRunner.run(JavascriptScriptRunner.java:39) at org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:228) ... 11 common frames omitted Caused by: java.lang.NoSuchMethodException: Can't unambiguously select between fixed arity signatures [(org.apache.nifi.logging.LogMessage), (java.lang.String)] of the method org.apache.nifi.controller.TerminationAwareLogger.info for argument types [null] at jdk.internal.dynalink.beans.OverloadedMethod.throwAmbiguousMethod(OverloadedMethod.java:225) at jdk.nashorn.internal.scripts.Script$299$\^eval\_.:program(<eval>:13) at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:637) at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:494) at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393) ... 18 common frames omitted Any idea how to fix this? I want to avoid checking null values for logging also Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
02-03-2022
09:13 AM
Hello Experts, I am looking for a regex to configure in "RouteOnContent" processor of Nifi so that based on specific string in flow file content I can route FF. FF content : ● rclone.service - Rclone(file transfer tool) Remote Service
Loaded: loaded (/etc/systemd/system/rclone.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-02-03 09:46:01 UTC; 6h ago
Main PID: 23647 (rclone)
Tasks: 13
Memory: 61.3M
CPU: 5.143s
CGroup: /system.slice/rclone.service
└─23647 /usr/bin/rclone rcd --rc-no-auth --rc-job-expire-duration=60m --cache-tmp-upload-path=/tmp/rclone/upload I want to search for the string "Active: active (running)" in flow file content, if present route to different path. I tried few regex but did not work as expected. Any suggestion would be much appreciated. Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
01-19-2022
08:35 AM
Hi @ckumar - thanks for the reply. I think that screenshot lacked info may be due to my permission issue. Same issue has been posted by team mate here - https://community.cloudera.com/t5/Support-Questions/NiFi-Node-showing-2-nodes-and-not-respecting-node-down-fault/td-p/334221 Please take a look at the screenshot attachment there which has more info. And i can login from all 3 nodes ui, and in all nodes it shows up only 1 & 2.
... View more
01-19-2022
07:34 AM
Hello Experts, I have a 3 node nifi cluster with host name as nifi-hatest-01, nifi-hatest-02, nifi-hatest-03. nifi-hatest-01 is primary node at the moment and I can see (in logs) it is receiving heartbeats from 02 & 03. And if I create flows in any one of the 3 nodes it gets sycned to all nodes. But the problem now is the 3rd node (nifi-hatest-03) does not show up in the "cluster" tab of Nifi UI. Any idea what could be the issue? any suggestion would be much appreciated. @MattWho @GangWar @smdas Thanks Mahendra
... View more
Labels:
- Labels:
-
Ambari Blueprints
-
Apache NiFi
09-20-2021
05:26 AM
Hello, We are running single node nifi in 8 core machine and configured 'Maxmum Timer driven thread count' as 300. Once in a while Nifi get totally stuck (wont accept flowfiles) and also no app log will generate till we restart nifi manually. So wondering if its due to nifi thread starvation... In below article 'NiFi Thread starvation' section its mention that system will stuck in resource competition cycle. https://community.cloudera.com/t5/Community-Articles/NiFi-HDF-Dataflow-Optimization-Part-2-of-2/ta-p/245113 So any idea if nifi needs more threads than system available core then will there be any chance of nifi to get stuck ? @Wynner Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
08-08-2021
11:51 PM
@Green_ - Can I have a custom appender configured in logback.xml to collect logs and put that in MongoDB ? would that be possible with nifi logs ?
... View more