Hello Experts,   I need to query delta table from apache nifi. Since Nifi cant connect delta table directly, I am trying it through Spark Thrift Server.   I have used 'Hive3ConnectionPool' controller service with 'SelectHive3QL' processor and used below jdbc url and it worked for me. jdbc:hive2://localhost:10002   But I dont know whether using jdbc:hive2//<host>:<port> is right approach or should I use jdbc:spark//<host>:<port> url? Will there be any performance difference between jdbc:hive2 & jdbc:spark jdbc url's with respect to performance or any other imp thing? I tried jdbc:spark//<host>:<port> url with 'DBCPConnectionPool 1.16.3' cntroller service but could not succeed with DatabricksJDBC jar. Any suggestion would be appreciated   Thanks Mahendra ... View more

Hello Experts,   I am using nifi-api to update the variable registry of specific processor groups. It was working as expected, but in recent time its taking too long (some time infinite) time to complete the variable registry update. However if I restart nifi and try again this var registry update api it works quickly as expected.   Is it because of increased number/size of variable registry or any resource crunch ? Could you please suggest a solution on this ? Any suggestion is much appreciated in advance.   Thanks Mahendra ... View more

Hi Experts,   2022-12-22 10:47:41,012 ERROR [Timer-Driven Process Thread-131] c.o.n.p.o.TrackRequestStatus TrackRequestStatus[id=687e8099-cd28-3c8b-f788-cf64094bd870] ~(5196, 5196, 20582) ~ 5ed6349b-7eb8-49ab-ad42-963c82fa8bc9 ~ 5196.11730922 ~ Error updating track request status - Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state.Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] No LLAP Daemons are runningVertex killed, vertexName=Reducer 3, vertexId=vertex_1671455880373_0219_136_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:2, Vertex vertex_1671455880373_0219_136_02 [Reducer 3] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1671455880373_0219_136_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:2, Vertex vertex_1671455880373_0219_136_01 [Reducer 2] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Map 1, vertexId=vertex_1671455880373_0219_136_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1671455880373_0219_136_00 [Map 1] killed/failed due to:DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. failedVertices:0 killedVertices:3 java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Dag received [DAG_TERMINATE, SERVICE_PLUGIN_ERROR] in RUNNING state.Error reported by TaskScheduler [[2:LLAP]][SERVICE_UNAVAILABLE] No LLAP Daemons are runningVertex killed, vertexName=Reducer 3, vertexId=vertex_1671455880373_0219_136_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:2, Vertex vertex_1671455880373_0219_136_02 [Reducer 3] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1671455880373_0219_136_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:2, Vertex vertex_1671455880373_0219_136_01 [Reducer 2] killed/failed due to:DAG_TERMINATED]Vertex killed, vertexName=Map 1, vertexId=vertex_1671455880373_0219_136_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to DAG_TERMINATED, failedTasks:0 killedTasks:1, Vertex vertex_1671455880373_0219_136_00 [Map 1] killed/failed due to:DAG_TERMINATED]DAG did not succeed due to SERVICE_PLUGIN_ERROR. failedVertices:0 killedVertices:3 at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:401) at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:266) at org.apache.hive.jdbc.HivePreparedStatement.execute(HivePreparedStatement.java:101) at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:94) at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:94) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254) at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.access$100(StandardControllerServiceInvocationHandler.java:38) at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:240) at com.sun.proxy.$Proxy412.execute(Unknown Source) at com.o9solutions.nifi.services.TrackRequestStatusHiveDatabaseServiceImpl.saveStatus(TrackRequestStatusHiveDatabaseServiceImpl.java:114) at com.o9solutions.nifi.processors.o9_custom_processors.TrackRequestStatus.onTrigger(TrackRequestStatus.java:286) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1283) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)   Thanks, Mahendra ... View more