Member since
12-03-2017
140
Posts
20
Kudos Received
11
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
591 | 11-03-2023 12:17 AM | |
2103 | 12-12-2022 09:16 PM | |
753 | 07-14-2022 03:25 AM | |
1261 | 07-28-2021 04:42 AM | |
1590 | 06-23-2020 10:08 PM |
12-22-2022
05:52 AM
Hi Experts, 2022-12-22 10:47:41,012 ERROR [Timer-Driven Process Thread-131] c.o.n.p.o.TrackRequestStatus TrackRequestStatus[id=687e8099-cd28-3c8b-f788-cf64094bd870] ~(5196, 5196, 20582) ~ 5ed6349b-7eb8-49ab-ad42-963c82fa8bc9 ~ 5196.11730922 ~ Error updating track request status - Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state.Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] No LLAP Daemons are runningVertex killed, vertexName=Reducer 3, vertexId=vertex_1671455880373_0219_136_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:2, Vertex vertex_1671455880373_0219_136_02 [Reducer 3] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1671455880373_0219_136_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:2, Vertex vertex_1671455880373_0219_136_01 [Reducer 2] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Map 1, vertexId=vertex_1671455880373_0219_136_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1671455880373_0219_136_00 [Map 1] killed/failed due to:DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. failedVertices:0 killedVertices:3 java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state.Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] No LLAP Daemons are runningVertex killed, vertexName=Reducer 3, vertexId=vertex_1671455880373_0219_136_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:2, Vertex vertex_1671455880373_0219_136_02 [Reducer 3] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1671455880373_0219_136_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:2, Vertex vertex_1671455880373_0219_136_01 [Reducer 2] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Map 1, vertexId=vertex_1671455880373_0219_136_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1671455880373_0219_136_00 [Map 1] killed/failed due to:DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. failedVertices:0 killedVertices:3 at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:401) at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:266) at org.apache.hive.jdbc.HivePreparedStatement.execute(HivePreparedStatement.java:101) at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:94) at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:94) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254) at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.access$100(StandardControllerServiceInvocationHandler.java:38) at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:240) at com.sun.proxy.$Proxy412.execute(Unknown Source) at com.o9solutions.nifi.services.TrackRequestStatusHiveDatabaseServiceImpl.saveStatus(TrackRequestStatusHiveDatabaseServiceImpl.java:114) at com.o9solutions.nifi.processors.o9_custom_processors.TrackRequestStatus.onTrigger(TrackRequestStatus.java:286) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1283) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
12-12-2022
09:16 PM
1 Kudo
@steven-matison - thank you. Found out the reason for NoClassdef error it was due to the missing nar dependency in my NAR generation project pom. Need to add "nifi-mongodb-nar" dependency to make use of these mongo related classes as they are not part of standard nar. But I am already using "nifi-standard-nar" as dependency in NAR project I cant use one more, so had drop down this idea and write my own Mongo controller service.
... View more
11-29-2022
11:25 PM
Hi @SirV can you please send me the code , m also stuck in the same situation.
... View more
11-22-2022
09:43 PM
@Green_ Thank you so much ! It worked as expected.
... View more
11-08-2022
12:01 PM
@hegdemahendra This is not something I have tried before, but... When you execute the nifi.sh script to start it bootstrap the NiFi process via the configuration in the bootstrap.conf NiFi configuration file. It is during the bootstrap process that NiFi starts the main child process that loads NiFi. Perhaps you can add additional java.args to handle your pre NiFi needs? Or maybe modify the the nifi.sh script itself so that is executes your requirements before calling the rest of the NiFi startup process. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
10-20-2022
12:29 AM
Hi @krishna123 Please try changing the port, you won't face the same issue.
... View more
10-17-2022
02:38 AM
Hi When 'Event Hub Namespace' is set with invalid, incoming flow files are getting stuck (not moving to failure relation) and also its keep on opening connection (files) and which is making Nifi is to go down.. Any suggestion how can I take care of this ? In my use case these 'Event Hub Namespace' and other fields are configured from an UI by an end customer so possibility of invalid data Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
08-28-2022
04:17 PM
@hegdemahendra , You need to set the time characteristic of the stream for it to work. For example, try setting it to processing time, as shown below: DataStream<String> matechedStream = patternStream
.inProcessingTime()
.process(new PatternProcessFunction<String, String>() {
@Override
public void processMatch(Map<String, List<String>> map, Context context, Collector<String> collector) throws Exception {
collector.collect(map.get("start").toString());
}
}
); Cheers, André
... View more
08-12-2022
03:55 AM
Hello Experts, Any idea why we cant increase cuncurrent task of ConsumeAzureEventHub processor? And what is main difference between GetAzureEventHub vs ConsumeAzureEventHub processors? When should I use which one? Any info is much appreciated. Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
08-01-2022
09:07 PM
1 Kudo
@MattWho - We have separated out ebs volumes of each repos and also 3 ebs volumes each for content and provenance repos. Now looks like issue is pretty much resolved ! Thanks for all your suggestions, helped a lot Thanks Mahendra
... View more