Hi everyone,
I have been facing intermittent issues with a delete statement on a iceberg table v2.
Below is the error I am having and I dont know what is causing it.
Had anyone gone through the same issue?
Thank you
[HiveServer2-Background-Pool: Thread-1255]: Vertex failed, vertexName=Reducer 2, vertexId=vertex_1752069399412_0193_1_01, diagnostics=[Task failed, taskId=task_1752069399412_0193_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Node: xxxxx-compute0.xxxxx.ysv060.a0.cloudera.site/10.XX.XX.XX : Error while running task ( failure ) : attempt_1752069399412_0193_1_01_000000_0:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing vector batch (tag=0) (vectorizedVertexNum 1)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:351)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:280)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:84)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:70)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:70)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:40)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing vector batch (tag=0) (vectorizedVertexNum 1)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecordVector(ReduceRecordSource.java:411)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:318)
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:297)
... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing vector batch (tag=0) (vectorizedVertexNum 1)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:519)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecordVector(ReduceRecordSource.java:402)
... 19 more
Caused by: java.lang.UnsupportedOperationException: writePage with SizeStatistics is not implemented
at org.apache.parquet.column.page.PageWriter.writePage(PageWriter.java:99)
at org.apache.parquet.column.impl.ColumnWriterV1.writePage(ColumnWriterV1.java:69)
at org.apache.parquet.column.impl.ColumnWriterBase.writePage(ColumnWriterBase.java:378)
at org.apache.parquet.column.impl.ColumnWriteStoreBase.sizeCheck(ColumnWriteStoreBase.java:242)
at org.apache.parquet.column.impl.ColumnWriteStoreBase.endRecord(ColumnWriteStoreBase.java:227)
at org.apache.parquet.column.impl.ColumnWriteStoreV1.endRecord(ColumnWriteStoreV1.java:29)
at org.apache.iceberg.parquet.ParquetWriter.add(ParquetWriter.java:139)
at org.apache.iceberg.deletes.PositionDeleteWriter.write(PositionDeleteWriter.java:64)
at org.apache.iceberg.deletes.PositionDeleteWriter.write(PositionDeleteWriter.java:35)
at org.apache.iceberg.io.RollingFileWriter.write(RollingFileWriter.java:90)
at org.apache.iceberg.io.ClusteredWriter.write(ClusteredWriter.java:104)
at org.apache.iceberg.mr.hive.writer.HiveIcebergDeleteWriter.write(HiveIcebergDeleteWriter.java:66)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:1176)
at org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.process(VectorFileSinkOperator.java:111)
at org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:968)
at org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:158)
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:503)
... 20 more
