question Re: Is there a limit on number of namenodes we can have in Hadoop-2.x cluster ? in Archives of Support Questions (Read Only)

Is there a limit on number of namenodes we can have in Hadoop-2.x cluster ?

vin15 — Fri, 16 Sep 2022 09:31:29 GMT

Re: Is there a limit on number of namenodes we can have in Hadoop-2.x cluster ?

GautamG — Mon, 15 Jun 2015 16:05:28 GMT

If HDFS Federation is involved, you can have up to two namenodes per namespace in a HA configuration. If your entire HDFS is using one namespace, then you can have up to Namenodes in the HA configuration.

Re: Is there a limit on number of namenodes we can have in Hadoop-2.x cluster ?

vin15 — Mon, 15 Jun 2015 18:40:01 GMT

If we need to configure HA with HDFS federation, do we need to have standby namenode for every active namenode in the cluster ?

Is there a link or document that i can refer to configure both HA and HDFS federation.. All that i have seen so far is either to configure HA or federation but not both.

Re: Is there a limit on number of namenodes we can have in Hadoop-2.x cluster ?

GautamG — Mon, 15 Jun 2015 18:51:40 GMT

I should have mentioned earlier, HDFS Federation is not really supported in CDH5/CM5. Federation is from a time period where Namenode performance wasn't very scalable. Recent releases of Hadoop are quite good at scaling to many hundreds of nodes.

Is there a reason you want to use Federation? Maybe we can resolve your problem another way.