Support Questions
Find answers, ask questions, and share your expertise

Spliting resources among service

Spliting resources among service

Hi All, 

 

Recently I deployed 4 node cluster with CDH 5.7.1 and hardware details are as follows :

 

RAM - 16Gb , Hard disk - 500Gb

 

Can someone tell me how to allocate resources to services, mostly I am using hive, spark and Hbase (I/O intense jobs).

 

Is there any pre-defined calculations on how to split resources ? Please share. 

 

Just FyI : I saw cloudera performance tuning sheet and it didn't help me much 

 

Thanks

Kishore 

3 REPLIES 3

Re: Spliting resources among service

Rising Star

Dear Kishore,

 

I think static service pools what you are looking for. This documentation can help you:

 

http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_service_pools.html

 

Best regards,

 

    Gabor

 

 

Re: Spliting resources among service

Hi Gabor, 

 

Thanks for the reply, I will go through this link. To make my request simply. I want to assgin values for Yarn parameters like :

 

yarn.scheduler.maximum-allocation-mb, yarn.nodemanager.resource.memory-mb, yarn.app.mapreduce.am.resource.cpu-vcores , yarn.app.mapreduce.am.resource.mb, yarn.app.mapreduce.am.resource.mb, etc .. 

 

One more thing, I am confused about parameter Heap to Container Size Ratio (in my case it is 0.8).

 

Thanks

Kishore

Re: Spliting resources among service

Rising Star

Hello Kishore,

 

Here is a YARN tuning guide/spreadsheet which can help you to configure these parameters:

 

https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_yarn_tuning.html

 

Gabor