NiFI Failed to update Hive for FlowFile; java.lang.ArithmeticException: / by zero

[#1]

Hi there,

I am using a NiFi flow to create an SQL statement dynamically and execute it with a PutHive3QL processor to delete records from a Hive table that supports ACID transactions.

PutHive3Streaming works correctly to write new records to the table and apparently the table is working fine and there are no credentials issues.

The error I am getting with PutHive3QL is puzzling me and I really do not understand what is going on.

Any help is appreciated!

11:32:04 CESTERROR5b0a3ca0-21d2-16cf-a426-a2b4518fb710
All NodesPutHive3QL[id=5b0a3ca0-21d2-16cf-a426-a2b4518fb710] Failed to update Hive for FlowFile[filename=f77e9d70-415b-4037-9b35-5376192d6d8c] due to Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1747050143707_0616_1_00, diagnostics=[Vertex vertex_1747050143707_0616_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: TEST_TABLE1 initializer failed, vertex=vertex_1747050143707_0616_1_00 [Map 1], java.lang.ArithmeticException: / by zero
	at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:190)
	at org.apache.tez.dag.app.dag.RootInputInitializerManager.lambda$runInitializer$3(RootInputInitializerManager.java:203)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
	at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializer(RootInputInitializerManager.java:196)
	at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializerAndProcessResult(RootInputInitializerManager.java:177)
	at org.apache.tez.dag.app.dag.RootInputInitializerManager.lambda$createAndStartInitializing$2(RootInputInitializerManager.java:171)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:75)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1747050143707_0616_1_01, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1747050143707_0616_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1; it is possible that retrying the operation will succeed, so routing to retry: java.sql.SQLException: Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1747050143707_0616_1_00, diagnostics=[Vertex vertex_1747050143707_0616_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: TEST_TABLE1 initializer failed, vertex=vertex_1747050143707_0616_1_00 [Map 1], java.lang.ArithmeticException: / by zero
	at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:190)
	at org.apache.tez.dag.app.dag.RootInputInitializerManager.lambda$runInitializer$3(RootInputInitializerManager.java:203)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
	at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializer(RootInputInitializerManager.java:196)
	at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInitializerAndProcessResult(RootInputInitializerManager.java:177)
	at org.apache.tez.dag.app.dag.RootInputInitializerManager.lambda$createAndStartInitializing$2(RootInputInitializerManager.java:171)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:75)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1747050143707_0616_1_01, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1747050143707_0616_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1

NiFi version is 1.18.0 as a project requirement, meaning there is no chance we will be updating it anytime soon.

[#2]

Hi, @linssab 

This error (java.lang.ArithmeticException: / by zero in HiveSplitGenerator) usually comes from Hive when the query compiles into an empty or invalid input split. With PutHive3QL, DELETE/UPDATE operations on ACID tables often trigger a full table scan, and if stats are missing or corrupted, Tez can fail this way. First, try running the same SQL directly in Hive CLI/Beeline to confirm it’s not NiFi-specific. Then, run ANALYZE TABLE <table> COMPUTE STATISTICS and ANALYZE TABLE <table> COMPUTE STATISTICS FOR COLUMNS to refresh stats. Also check that the table is bucketed/transactional as required for ACID.

[#3]

Is the source table a JdbcStorageHandler table?

Please provide the DDL of the source table, the query used, and any sample data if possible. This information will help us understand the problem better.

Also, validate the set -v command, especially configurations like hive.tez.container.size.