Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Corrupted records with different bucket ids from the containing bucket file found!

Corrupted records with different bucket ids from the containing bucket file found!

New Contributor

step:

  1. create partition and bucket table
  2. insert into data by partition
  3. delete one partition data from this table
  4. repeat insert into data to this partition select from other table (data is more than before delete)
  5. now we delete data from this partition ,we get the error:
Caused by: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found the bucket id 1 from owid: 1 bucketP:536936448 rowid: 0 curTxnId: 2 at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1029) at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:971) at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1213) at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1156) at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:224) at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:121) at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:116) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1984) at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:417)
1 REPLY 1

Re: Corrupted records with different bucket ids from the containing bucket file found!

New Contributor

any one know why?

Don't have an account?
Coming from Hortonworks? Activate your account here