question Re: what is the best practice for scale out the servers, what cpu. memory and disk space is recommended per each server in Archives of Support Questions (Read Only)

what is the best practice for scale out the servers, what cpu. memory and disk space is recommended per each server

Sanaz_janbakhsh — Tue, 21 Apr 2026 12:24:30 GMT

Hi,

Based on the posts in the community, looks like physical servers are recommended rather than VM cluster.

Now i would like to know what is the best practice for scale out the servers and what memory, cpu and disk is recommended .

Looks like it is also recommended to scale out (multiple machines) rather than scale up ( smaller number of large machines).

Can you please advise more details about the best practice for scale out the HDP and HDF servers?

Thanks,

Re: what is the best practice for scale out the servers, what cpu. memory and disk space is recommended per each server

jsensharma — Thu, 09 Mar 2017 14:31:38 GMT

@Sanaz Janbakhsh

It depends on various factors like the kind of components that you are planning to use, the kind of job and the amount of data that you are planning to process, size of cluster ...etc.

1. But in general you can refer to the following doc to get a good idea (Cluster Planning): http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_cluster-planning/content/ch_hardware-recommendations_chapter.html

2. Similarly for Slave/Master recommendations: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_cluster-planning/content/hardware-for-slave.1.html

3. Also you can get some idea about the Memory requirement for NameNode based on the number of files: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_command-line-installation/content/ref-80953924-1cbf-4655-9953-1e744290a6c3.1.html

4. You can also use the HDP utility script which is the recommended method for calculating HDP memory configuration settings: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_command-line-installation/content/determine-hdp-memory-config.html

Re: what is the best practice for scale out the servers, what cpu. memory and disk space is recommended per each server

Sanaz_janbakhsh — Fri, 10 Mar 2017 15:50:18 GMT

Thanks Jay SenSharma,

How about HDF?

SJ.

Re: what is the best practice for scale out the servers, what cpu. memory and disk space is recommended per each server

jsensharma — Thu, 23 Mar 2017 09:21:10 GMT

@Sanaz Janbakhsh

For HDF i responded to another thread just now.

https://community.hortonworks.com/questions/90400/minimum-hardware-and-clustering-requirements-for-h.html#answer-90414