Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spliting resources among service

Highlighted

Spliting resources among service

Hi All, 

 

Recently I deployed 4 node cluster with CDH 5.7.1 and hardware details are as follows :

 

RAM - 16Gb , Hard disk - 500Gb

 

Can someone tell me how to allocate resources to services, mostly I am using hive, spark and Hbase (I/O intense jobs).

 

Is there any pre-defined calculations on how to split resources ? Please share. 

 

Just FyI : I saw cloudera performance tuning sheet and it didn't help me much 

 

Thanks

Kishore 

3 REPLIES 3

Re: Spliting resources among service

Rising Star

Dear Kishore,

 

I think static service pools what you are looking for. This documentation can help you:

 

http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_service_pools.html

 

Best regards,

 

    Gabor

 

 

Re: Spliting resources among service

Hi Gabor, 

 

Thanks for the reply, I will go through this link. To make my request simply. I want to assgin values for Yarn parameters like :

 

yarn.scheduler.maximum-allocation-mb, yarn.nodemanager.resource.memory-mb, yarn.app.mapreduce.am.resource.cpu-vcores , yarn.app.mapreduce.am.resource.mb, yarn.app.mapreduce.am.resource.mb, etc .. 

 

One more thing, I am confused about parameter Heap to Container Size Ratio (in my case it is 0.8).

 

Thanks

Kishore

Re: Spliting resources among service

Rising Star

Hello Kishore,

 

Here is a YARN tuning guide/spreadsheet which can help you to configure these parameters:

 

https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_yarn_tuning.html

 

Gabor