Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Restricting Region Server memory on single host

Restricting Region Server memory on single host

New Contributor

Hi Experts

 

My HBase setup consists of 4 hosts . This setup consists of 3 hosts where region servers are installed.  I have 64 GB memory allocated to 2 hosts on which region servers are running, however one region server host has 32 GB memory only and it is not possible to increase memory in this host.  Now , what I want do is instruct HBASE and Yarn to limit memory only on the host which is having 32 GB memory. I want Yarn and HBASE to uses 64 GB memory on 2 hosts but 32 GB on one host. Is this something doable ?. What all parameters I need to change and how can I ensure that this parameters take effect only on single host ?

 

thanks in advance

-R

1 REPLY 1
Highlighted

Re: Restricting Region Server memory on single host

Expert Contributor

this sounds like a test/dev cluster, so i would recommend just making it easy on yourself and running just YARN on this smaller node. I would wager that this 32GB node is also older and likely has less CPU performance as well? this will make your hbase performance actually worse typically, since a table will be spread among all regionservers and the table scan will take longer there. Additionally HBase doesn't have many easy built in ways to give reduced amounts of regions to a specific regionserver. The balancer framework does allow you to write and specify your own hbase balancer, but this is a significant amount of work.

 

as far as what settings you'll need to adjust for YARN, that is covered here: https://www.cloudera.com/documentation/enterprise/5-3-x/topics/cdh_ig_yarn_tuning.html

 

Don't have an account?
Coming from Hortonworks? Activate your account here