Hadoop Cluster Configuration Rules

[#1]

Is is necessary that all the slaves in cluster must have same configuration??

[#2]

@Abhimanyu Dasarwar

When we talk of slaves in HDP usually we are referring to data nodes. In a good setup, you have the (Master, Slave and Edge node) see this link for reference.

The Master usually run very important components like Namenode, YARN, HBase, MR etc and should not be hosted on commodity hardware contrary to the slaves aka worker nodes can run on commodity hardware.

To answer your question slaves MUST not necessarily run on the same hardware specs [Storage, Memory ,VCores or CPU's] YARN will aggregate the different RAM,CPU etc available for the cluster and present it as a unified resource available for the tasks on the cluster.

HTH