ambari cluster + HW minimal requirements for production cluster

[#1]

hi all

typical production cluster should be as the following structure ( based on hadoop 2.6.5 version and ambari 2.6.1 version)

3 master machines ( name node machines , while aambari server installed on master02 machine and active stasndby name node are master01/03 )

3 kafka machines

5-250 workers machines ( data node machines )

we want to know what are the minimal HW requirements for master machines ?

memory ?

CPU ?

/var partition size ?

/ partition size ?

swap ?

[#2]

Hi @Michael Bronson

You can calculate the requirements based upon the amount and type of data using the following guide:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.3/bk_cluster-planning/bk_cluster-planning.pdf

[#3]

Hi @Michael Bronson

I don't agree with type of nodes, you should have 03 :

- 02 Admin machines (at least 8 cores / 32 Go RAM) : Ambari / Ranger / ETC

- 03 Master machines (at least 8 cores / 64 Go RAM): NN / NN HA / Hive Server and others master services

- workers machines (at least 16 cores / 128 Go RAM)

@Sparsh Singhal point you to a good document have a look at it too.

[#4]

what about partitions size for /var and / ? and swap policy ?

[#5]

@Michael Bronson

Document has recommended partition size for / and /var. /var mostly has your logs which can usually take up a lot of space. AFAIK swap should be disabled and swappiness should be 0.