Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Can you have multiple (more than 1) secondary NameNode when configuring HDFS HA?,

avatar
Explorer

I'm setting up NameNode HA on a cluster and we'd like to have more than 1 secondary NameNode. Looking at the procedures for setting it up it doesn't look like it is supported:

Via Ambari:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_Ambari_Users_Guide/content/_how_to_confi...

Manually:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_hadoop-ha/content/ha-nn-config-cluster.h...

,

1 ACCEPTED SOLUTION

avatar
Super Guru
@John Jackson

This feature is currently not available but will be shipped with Hadoop 3.0. You will be able to have up to 5 name nodes. Please see the following JIRA.

https://issues.apache.org/jira/browse/HDFS-6440

View solution in original post

4 REPLIES 4

avatar
Master Guru

@John Jackson I am not aware of any capabilities for multipule secondary namenodes. If that would be possible then I assume enhancements to the journal keeper service will need to be made to keep multipule instances in sync. Would like to see others opinion if that is possible and supported. To the best of my knowledge it is not. One possibility you have is HDFS federation. where you can have multipule name nodes and each once having secondary name node. Each namenode would be responsible for slice of your namespace. here is the architecture

Multiple Namenodes/Namespaces

In order to scale the name service horizontally, federation uses multiple independent Namenodes/namespaces. The Namenodes are federated; the Namenodes are independent and do not require coordination with each other. The Datanodes are used as common storage for blocks by all the Namenodes. Each Datanode registers with all the Namenodes in the cluster. Datanodes send periodic heartbeats and block reports. They also handle commands from the Namenodes.

Users may use ViewFs to create personalized namespace views. ViewFs is analogous to client side mount tables in some Unix/Linux systems.

HDFS Federation Architecture

More on federation here.

avatar
Explorer

I had looked at this earlier, and while compelling, doesn't quite meet our needs. Thanks for the info!

avatar
Super Guru
@John Jackson

This feature is currently not available but will be shipped with Hadoop 3.0. You will be able to have up to 5 name nodes. Please see the following JIRA.

https://issues.apache.org/jira/browse/HDFS-6440

avatar
Explorer

This was exactly what I was looking for. Thanks for the heads up!