Support Questions

Find answers, ask questions, and share your expertise
Celebrating as our community reaches 100,000 members! Thank you!

What is causing Hive tables to become locked?

Rising Star

Hi all,

Bit of background, I have been struggling with an issue in which a table can get locked but the lock gets stuck in a "waiting" state and so does not allow the query to progress, after about 2 hours the job complains that the table is locked and it cannot access it. I'm trying to identify why the table is getting locked which brings me to my question....

If a user runs a query but then disconnects the session can this cause a table to get locked in this state?

| lockid | database | table | partition | lock_state | blocked_by | lock_type | transaction_id | last_heartbeat | acquired_at | user | hostname | agent_info

| 78011111.2 | db | tbl | NULL | WAITING | 78043210.2 | EXCLUSIVE | NULL | 1523459564452 | NULL | user | host | hive_20180411161225_2b33e811-e44c-59ds-afb3-b4111fcb019a |


Super Collaborator property enables locking. When a queries is shutdown its locks should be released immediately. When dies abruptly it may leave locks behind. These will be cleaned up by a background process running from a standalone Hive metastore process. This process will consider locks abandoned if they have not heartbeated for (by default) 5 minutes. Metastore logfile should have entries from AcidHouseKeeperService - that is the clean up process.