Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What shall be the Tez container size if different Hosts configuration exists

Highlighted

What shall be the Tez container size if different Hosts configuration exists

New Contributor

Hi

I am using HDP 2.6.4.0 and have 300+ hosts with below configs:

Few hosts with 32 cores & 250 GB RAM

Few with 48 cores & 250 GB RAM

Others with 24 cores & 180 GB RAM

Now I read somewhere that we should not have more than 1 container per processor. So with this recommendation, I shall get ~7GB, ~5GB & ~7GB respectively as tez.container.size to maximize utilization of memory.

Now if I am setting "hive.tez.container.size" as 7 GB does it mean I am wasting space in hosts (48 cores/250 GB RAM) ?

What shall be the ideal TEZ container size in such case.


Thanks in advance.

Raj