Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hive metastore load balancing

hive metastore load balancing

New Contributor

hi. i have a small question.

 

i'm using 2 hiveserver and 2 hive metastore.

- hive-server-01 : hiveserver2 and metastore

- hive-server-02 : hiveserver2 and metastore

 

can i set configuration this?

hiveserver2 (hive-server-01)

- hive.metastore-uris : thrift://hive-server-01,thrift://hive-server-02

 

hiveserver2 (hive-server-02)

- hive.metastore-uris : thrift://hive-server-02,thrift://hive-server-01

 

using different metastore(master).

 

 

thank

@hyunwoo

1 REPLY 1
Highlighted

Re: hive metastore load balancing

Contributor
Yes, that configuration should work.

But you should we aware that you now have 2 instances that can modify the hive metadata without locking at the hive level. Meaning, the database is the one that is in-charge of concurrency.

Say you have a HMS client on one reading the full HMS snapshot via one instance, and at the same time, another client is modifying the metadata via the other instance, you may (or may not) see exceptions in the log about certain cached objects (in datanucleus) were no longer found. The HMS handler has retrying logic in it that will retry the transaction and succeed.
It is safe to ignore the exceptions as long as the queries are succeeding.

Hope this helps. Thanks