typical production cluster should be as the following structure ( based on hadoop 2.6.5 version and ambari 2.6.1 version)
3 master machines ( name node machines , while aambari server installed on master02 machine and active stasndby name node are master01/03 )
3 kafka machines
5-250 workers machines ( data node machines )
we want to know what are the minimal HW requirements for master machines ?
/var partition size ?
/ partition size ?
You can calculate the requirements based upon the amount and type of data using the following guide:
I don't agree with type of nodes, you should have 03 :
- 02 Admin machines (at least 8 cores / 32 Go RAM) : Ambari / Ranger / ETC
- 03 Master machines (at least 8 cores / 64 Go RAM): NN / NN HA / Hive Server and others master services
- workers machines (at least 16 cores / 128 Go RAM)
@Sparsh Singhal point you to a good document have a look at it too.