Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)

SYMPTOM:

Problem was seen when the client was running Metastore HA and both configured with org.apache.hadoop.hive.ql.txn.compactor.Initiator. this is what we observed from the logs of both metastore service

// metastore 1
 ERROR compactor.Worker (Worker.java:run(181)) - Caught an exception in the main loop of compactor worker lnxhdpap02.smrcy.com-33, MetaException(message:Unable to connect to transaction database com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction 

// metastore 2
 ERROR txn.CompactionTxnHandler (CompactionTxnHandler.java:findNextToCompact(194)) - Unable to select next element for compaction, Deadlock found when trying to get lock; try restarting transaction 

ROOT CAUSE:

org.apache.hadoop.hive.ql.txn.compactor.Initiator is a thread that runs inside the metastore service to manage compactions of ACID tables. There should be exactly 1 instance of this thread (even with multiple Thrift services).

WORKAROUND:

NA

RESOLUTION:

set “hive.compactor.initiator.on" only for single instance of metastore service.

825 Views
Comments
New Contributor

@Rajkumar Singh - this is a useful article and I've probably passed it on to 5-6 customers. One thing to note though, this is no longer an issue with HDP3. :) This is still very applicable and necessary for HDP2.x.

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎12-21-2016 07:11 PM
Updated by:
 
Contributors
Top Kudoed Authors