I am using HDP 184.108.40.206 and have 300+ hosts with below configs:
Few hosts with 32 cores & 250 GB RAM
Few with 48 cores & 250 GB RAM
Others with 24 cores & 180 GB RAM
Now I read somewhere that we should not have more than 1 container per processor. So with this recommendation, I shall get ~7GB, ~5GB & ~7GB respectively as tez.container.size to maximize utilization of memory.
Now if I am setting "hive.tez.container.size" as 7 GB does it mean I am wasting space in hosts (48 cores/250 GB RAM) ?
What shall be the ideal TEZ container size in such case.