Support Questions

Find answers, ask questions, and share your expertise

ORC acid table "select count(*) error" - [Error 30021]: An ORC ACID reader required to read ACID tables

I created a table as below. Then ingest date by rk and dt. After two separate insert into, when I run a select count(*), I keep getting the following error: "Caused by: [Error 30021]: An ORC ACID reader required to read ACID tables"

What does this error mean, how do I work around this?

set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;

CREATE TABLE `table1_orc` ( `uuid` string, token string, ip_address string, `raw_event` string ) PARTITIONED BY ( `rk` string, `dt` string ) CLUSTERED BY (token) INTO 10 BUCKETS STORED AS ORC TBLPROPERTIES ( 'transactional'='true');


Finally figured out.need to set;

View solution in original post


Expert Contributor


"Reading/writing to an ACID table from a non-ACID session is not allowed. In other words, the Hive transaction manager must be set to org.apache.hadoop.hive.ql.lockmgr.DbTxnManager in order to work with ACID tables."

SET hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;

hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager is set in the configuration indeed. I run the select count(*) from the same session that "insert into" was done. "insert into" was fine.

I created a similar acid table but without partition. Query runs fine. So it seems something related to partition.

--1st error is: "Caused by: [Error 30022]: Must use HiveInputFormat to read ACID tables (set".

--After I set set, I got second error: "Caused by: [Error 30021]: An ORC ACID reader required to read ACID tables".

--Set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager, make no difference

--I tried compact table all partitions major, still get same error. Any suggestion for working around the error is appreciated.

Here the table definition:
CREATE TABLE `raw_orc1` ( `uuid` string, token string, ip string, event string ) PARTITIONED BY ( `rk` string, `dt` string ) CLUSTERED BY (token) INTO 10 BUCKETS STORED AS ORC TBLPROPERTIES ( 'transactional'='true' );

at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor( at at at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$ at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$ at Method) at at at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal( at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal( at at at java.util.concurrent.ThreadPoolExecutor.runWorker( at java.util.concurrent.ThreadPoolExecutor$ at Caused by: java.lang.reflect.InvocationTargetException at at at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader( at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>( at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader( at at org.apache.tez.mapreduce.lib.MRReaderMapred.setupOldRecordReader( at org.apache.tez.mapreduce.lib.MRReaderMapred.setSplit( at org.apache.tez.mapreduce.input.MRInput.initFromEventInternal( at org.apache.tez.mapreduce.input.MRInput.initFromEvent( at org.apache.tez.mapreduce.input.MRInputLegacy.checkAndAwaitRecordReaderInitialization( at org.apache.tez.mapreduce.input.MRInputLegacy.init( at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getMRInput( at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init( at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor( ... 14 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance( at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( at java.lang.reflect.Constructor.newInstance( at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader( ... 26 more Caused by: [Error 30021]: An ORC ACID reader required to read ACID tables at at$VectorizedOrcRecordReader.<init>( at at at at<init>( ... 31 more

Finally figured out.need to set;