question Re: Can you have multiple (more than 1) secondary NameNode when configuring HDFS HA?, in Archives of Support Questions (Read Only)

Can you have multiple (more than 1) secondary NameNode when configuring HDFS HA?,

jacks_john — Thu, 04 Aug 2016 23:50:41 GMT

I'm setting up NameNode HA on a cluster and we'd like to have more than 1 secondary NameNode. Looking at the procedures for setting it up it doesn't look like it is supported:

Via Ambari:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_Ambari_Users_Guide/content/_how_to_configure_namenode_high_availability.html

Manually:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_hadoop-ha/content/ha-nn-config-cluster.html

Re: Can you have multiple (more than 1) secondary NameNode when configuring HDFS HA?,

sunile_manjee — Fri, 05 Aug 2016 00:23:44 GMT

@John Jackson I am not aware of any capabilities for multipule secondary namenodes. If that would be possible then I assume enhancements to the journal keeper service will need to be made to keep multipule instances in sync. Would like to see others opinion if that is possible and supported. To the best of my knowledge it is not. One possibility you have is HDFS federation. where you can have multipule name nodes and each once having secondary name node. Each namenode would be responsible for slice of your namespace. here is the architecture

Multiple Namenodes/Namespaces

In order to scale the name service horizontally, federation uses multiple independent Namenodes/namespaces. The Namenodes are federated; the Namenodes are independent and do not require coordination with each other. The Datanodes are used as common storage for blocks by all the Namenodes. Each Datanode registers with all the Namenodes in the cluster. Datanodes send periodic heartbeats and block reports. They also handle commands from the Namenodes.

Users may use ViewFs to create personalized namespace views. ViewFs is analogous to client side mount tables in some Unix/Linux systems.

Re: Can you have multiple (more than 1) secondary NameNode when configuring HDFS HA?,

mqureshi — Fri, 05 Aug 2016 01:39:55 GMT

@John Jackson

This feature is currently not available but will be shipped with Hadoop 3.0. You will be able to have up to 5 name nodes. Please see the following JIRA.

https://issues.apache.org/jira/browse/HDFS-6440

Re: Can you have multiple (more than 1) secondary NameNode when configuring HDFS HA?,

jacks_john — Fri, 05 Aug 2016 03:37:00 GMT

This was exactly what I was looking for. Thanks for the heads up!

Re: Can you have multiple (more than 1) secondary NameNode when configuring HDFS HA?,

jacks_john — Fri, 05 Aug 2016 03:38:33 GMT

I had looked at this earlier, and while compelling, doesn't quite meet our needs. Thanks for the info!