Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive Mysql Metastore Row lock increases

avatar
New Contributor

Below is the query causing row locks in Mariadb Galera Cluster 10.2.32 (3Node Cluster).

 

select MT_COMMENT from AUX_TABLE where MT_KEY1='Cleaner' and MT_KEY2=0 for update

 

> Hive is a framework, we couldn't get the flow to understand also can't alter the logic from app end.

 

> Looking for solutions like any version that works in Mysql/Postgresql or any Metastore DBs that can scale for Hive Metastore without lockign issues.

 

> Also looking for any work around solution from DB end for this Mysql Row Locking.

5 REPLIES 5

avatar
Community Manager

Welcome to the community @Mannoj. I'm not an expert but while you wait for one, I found this article/video on Hive Locking Mechanism. Hopefully it moves you closer in the right direction. 


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
New Contributor

@cjervis These are locks in Hive, we have a mysql backend for metastore and that keeps locking in above mentioned query. Whenever the row locks increase at mysql end we will face issues in hive in processing queries and backlog builds.

avatar
Community Manager

Thank you for the clarification. Hopefully someone will be able to assist.


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Master Collaborator

@Mannoj RDBMS HA for metastore is not officially supported yet. We have a knowledge article on the same. Cloudera's statement on RDBMS HA can be found here.

avatar
New Contributor
I dont have cloudera enterprise login. So can’t check. Such KA can be open
to public.