Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive read write locks

Highlighted

Hive read write locks

Expert Contributor

I have scheduled Hive queries using oozie that perform INSERT OVERWRITE on tables to update the data. I have been seeing this since some time that a lot of queries fail to update data because of locks with ERROR : Failed to acquire readWrite locks. It is important for these queries to update the data at scheduled time and it fails to do it because of this read write locking thing a lot of times. How do I prevent this from happening? I need to have a reliable system so that Hive queries are updating data at scheduled time always without FAIL.

2 REPLIES 2
Highlighted

Re: Hive read write locks

Rising Star

Can you post the exception in the HiveServer2 logs? Which version of Hive/HDP are you running? You might consider about upgrading to a newer version of Hive which has a better concurrency handling around acquiring read/write locks.

Highlighted

Re: Hive read write locks

Expert Contributor

@dsun: Here's the error I get with INSERT OVERWRITE statement

Log Type: stdout

Log Upload Time: Fri Mar 17 11:20:01 +0530 2017

Log Length: 606746

Heart beat
Heart beat
Heart beat
Heart beat
6712698 [main] ERROR org.apache.hadoop.hive.ql.Driver  - FAILED: Error in acquiring locks: Locks on the underlying objects cannot be acquired. retry after some time
org.apache.hadoop.hive.ql.lockmgr.LockException: Locks on the underlying objects cannot be acquired. retry after some time
	at org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager.acquireLocks(DummyTxnManager.java:164)
	at org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:1025)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1301)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1120)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:218)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:170)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:381)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:316)
	at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:414)
	at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:430)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:724)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:691)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:626)
	at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:325)
	at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:302)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:49)
	at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:69)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Don't have an account?
Coming from Hortonworks? Activate your account here