Reply
Highlighted
New Contributor
Posts: 1
Registered: ‎08-15-2018

hive metastore load balancing

hi. i have a small question.

 

i'm using 2 hiveserver and 2 hive metastore.

- hive-server-01 : hiveserver2 and metastore

- hive-server-02 : hiveserver2 and metastore

 

can i set configuration this?

hiveserver2 (hive-server-01)

- hive.metastore-uris : thrift://hive-server-01,thrift://hive-server-02

 

hiveserver2 (hive-server-02)

- hive.metastore-uris : thrift://hive-server-02,thrift://hive-server-01

 

using different metastore(master).

 

 

thank

@hyunwoo

Cloudera Employee
Posts: 34
Registered: ‎08-16-2016

Re: hive metastore load balancing

Yes, that configuration should work.

But you should we aware that you now have 2 instances that can modify the hive metadata without locking at the hive level. Meaning, the database is the one that is in-charge of concurrency.

Say you have a HMS client on one reading the full HMS snapshot via one instance, and at the same time, another client is modifying the metadata via the other instance, you may (or may not) see exceptions in the log about certain cached objects (in datanucleus) were no longer found. The HMS handler has retrying logic in it that will retry the transaction and succeed.
It is safe to ignore the exceptions as long as the queries are succeeding.

Hope this helps. Thanks
Announcements