Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Best balance of number of CPU and number of Disks for slave nodes

Highlighted

Best balance of number of CPU and number of Disks for slave nodes

New Contributor

I know we have best practice for balance of # of cores and # of disks.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_command-line-installation/content/determ...

# of containers = min (2*CORES, 1.8*DISKS, (Total available RAM) / MIN_CONTAINER_SIZE)

I believe, it means that slave nodes in "2*CORES = 1.8*DISKS" are best balanced one in point of CPUs and Disks.

Does anyone know the "CORES" means whether number of "physical" cores or number of "virtual" cores (i.e. Hyper-Threading Technology)?

If it means "physical" cores, number of physical CPU cores is nice to be 12 with 12 disks.

If it means "virtual" cores by for example Intel HT, 6 physical cores would be enough with 12 disks (best balanced node).

Also, I'm wandering, we should enable Hyper-Threading or not to get better "throughput".

Any reply, comment and suggestion will help me. Thanks!

1 REPLY 1

Re: Best balance of number of CPU and number of Disks for slave nodes

New Contributor

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_command-line-installation/content/determ...

This page also says "CORES (number of CPU cores)".

So, "CORES" means "physical cores"?

If it's true, 12 physical CPU cores (24 vcores by Intel HT) may be good enough for a node with 12 HDDs.

I'm welcome any opinions from all of you :)

Don't have an account?
Coming from Hortonworks? Activate your account here