Hadoop cluster failover

[#1]

Hi All,

I'm interested by any paper or returns of experience regarding the Hadoop cluster fail over design.

From my understanding 2 mains components concerned are NameNode and ResourceManager.

Thanks

Regards

Farhad

[#2]

Hi,

I have some questions about the Hadoop Cluster data node failover:

• What happened the link is down between the name node and a data node (or between 2 data nodes) when the Hadoop cluster is processing some data? Does Hadoop cluster have any OOTB to recover this problem?
• What happens one data node is down when the Hadoop cluster is processing some data?

Also, another question is about the Hadoop cluster hardware configuration. Let's say we will use our Hadoop cluster to process 100GB log files each day, how many data nodes do we need to set up? And for each data node hardware configuration(e.g. CPU, RAM, Harddisk)?

Thank You

Hari

[#3]

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.3/bk_cluster-planning/bk_cluster-planning.pdf

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cluster-planning/content/ch_hardware-rec...