question HDFS High Availability - is the Active & standby NN must have the same hardware configuration!?! in Archives of Support Questions (Read Only)

HDFS High Availability - is the Active & standby NN must have the same hardware configuration!?!

AcharkiMed — Fri, 16 Sep 2022 13:19:11 GMT

Hi,

I found this phrases in the HDFS High Availability (HA) documentation, and I want to know if it obligatory or it's just a recommendation !!!?

source: https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cdh_hag_hdfs_ha_hardware_config.html

NameNode hosts - These are the hosts on which you run the active and standby NameNodes. They should have equivalent hardware to each other

the host that is chosen should have the same hardware configuration (RAM, disk space, number of cores, and so on) as the active NameNode.

source: https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cdh_hag_hdfs_ha_enabling.html#cmug_topic_5_12

Thanks in adcance.

Re: HDFS High Availability - is the Active & standby NN must have the same hardware configuratio

csguna — Sat, 09 Jun 2018 07:00:43 GMT

Atleast most of the production cluster that i have seen have the unformity interms of hardware configurations and their version , same Vcores , Hard disk is madatory . Because you basically have a copy of the active namenode in the stand by . hope it helps .

Re: HDFS High Availability - is the Active & standby NN must have the same hardware configuratio

saranvisa — Sat, 09 Jun 2018 08:46:09 GMT

@AcharkiMed

Basically this is a recommendation, still it will be considered as Mandatory or Optional depends upon the environment that you are using

Ex:
*) For Prod Env - It is mandatory. Otherwise it will create performance difference if there is any switch between active and standby NN

*) For test/POC Env - It is optional if you don't have right choice

Re: HDFS High Availability - is the Active & standby NN must have the same hardware configuratio

AcharkiMed — Sat, 09 Jun 2018 14:54:03 GMT

Hi @saranvisa and @csguna
Thanks for your answers,

But I still must to make a disicion about configuring a HDFS HA or not!
My quesion is, if I will configure the HDFS HA with 2 diferent NN is it WORK or NOT ?!
For performence I'll not have a problem because the two servers are very strong (in fact all the ressources in the two hosts far exceeds the recommendations configurations !).

Thanks.

Re: HDFS High Availability - is the Active

Harsh J — Sun, 10 Jun 2018 04:45:35 GMT

It is a recommendation based on the fact that active and standby are merely
states of the NameNode and not different daemons.

The NameNode doesn't check it's own hardware to be the same as other
NameNodes if that's what you are worried about.