Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive ACID table compaction : error log in metastore

Hive ACID table compaction : error log in metastore

New Contributor


My sandbox (HDP2.5) shows errors in hive metastore logs :

2016-08-09 10:54:20,068 ERROR Unable to select next element for compaction, Unknown column 'cq_tblproperties' in 'field list'
2016-08-09 10:54:20,069 ERROR Caught an exception in the main loop of compactor worker, MetaException(message:Unable to connect to transaction database com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 'cq_tblproperties' in 'field list' at sun.reflect.GeneratedConstructorAccessor31.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( at java.lang.reflect.Constructor.newInstance( at com.mysql.jdbc.Util.handleNewInstance( at com.mysql.jdbc.Util.getInstance( at com.mysql.jdbc.SQLError.createSQLException( at com.mysql.jdbc.MysqlIO.checkErrorPacket( at com.mysql.jdbc.MysqlIO.checkErrorPacket( at com.mysql.jdbc.MysqlIO.sendCommand( at com.mysql.jdbc.MysqlIO.sqlQueryDirect( at com.mysql.jdbc.ConnectionImpl.execSQL( at com.mysql.jdbc.ConnectionImpl.execSQL( at com.mysql.jdbc.StatementImpl.executeQuery( at com.jolbox.bonecp.StatementHandle.executeQuery( at org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findNextToCompact( at ) at org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findNextToCompact( at

I expect it is not good for ACID table compactions. Might it be a regression since HDP 2.4 ?



Re: Hive ACID table compaction : error log in metastore

It seems update is not happen to data store that is why column(cq_tblproperties) is missing.

can you try adding below property and check if it helps.


Re: Hive ACID table compaction : error log in metastore

New Contributor

Thank you for your answer but that property is already set, as you can see below :



Re: Hive ACID table compaction : error log in metastore

Expert Contributor

Can you do a 'describe table COMPACTION_QUEUE' in the MySQL backing the metastore? That is the table that should have 'cq_tblproperties'. And yes, Compactions won't run because of this error.

Also please include "select * from VERSION" from the same DB.

Don't have an account?
Coming from Hortonworks? Activate your account here