Skip to main content
Daggers
Daggers
Silver ⭐️⭐️⭐️
February 21, 2019
Solved

Load balancer for metastore

Hi,

 

We have HA enabled for the metastore and have 2 instances running.  Is it possible to configre a load balancer in front of these ? and if so is this normal practice ?

 

Thanks

 

EricL
EricLAnswer
Platinum ⭐️⭐️
March 6, 2019
HMS does not currently support Load Balancing. You can only specify them in order in hive-site.xml and it will fall to the second HMS only when first HMS is not reachable.
View in thread

2 replies

EricL
Platinum ⭐️⭐️
EricLAnswer
Platinum ⭐️⭐️
March 6, 2019
HMS does not currently support Load Balancing. You can only specify them in order in hive-site.xml and it will fall to the second HMS only when first HMS is not reachable.
Translate
    noamh
    noamh
    Silver ⭐️⭐️
    March 25, 2019

    Hi,

     

    So how do you scale Hive Metastore when you need to support more than 80 active connections?

    According to Cloudera's documentation they only mention supporting up to 80 connections which needs 16-24GB.

    Does passing 80 connection requires to continue extending the Heap Memory? How big can it be since 24GB is already too big isn't it?

     

    Attaching Cloudera's memory setup recommendations:

    https://www.cloudera.com/documentation/enterprise/5-15-x/topics/cdh_ig_hiveserver2_configure.html

     

    Best, Eyal

    Translate
    Terms & ConditionsAccessibility statement