Created 08-04-2016 04:50 PM
I'm setting up NameNode HA on a cluster and we'd like to have more than 1 secondary NameNode. Looking at the procedures for setting it up it doesn't look like it is supported:
Via Ambari:
Manually:
,
Created 08-04-2016 06:39 PM
This feature is currently not available but will be shipped with Hadoop 3.0. You will be able to have up to 5 name nodes. Please see the following JIRA.
Created 08-04-2016 05:23 PM
@John Jackson I am not aware of any capabilities for multipule secondary namenodes. If that would be possible then I assume enhancements to the journal keeper service will need to be made to keep multipule instances in sync. Would like to see others opinion if that is possible and supported. To the best of my knowledge it is not. One possibility you have is HDFS federation. where you can have multipule name nodes and each once having secondary name node. Each namenode would be responsible for slice of your namespace. here is the architecture
In order to scale the name service horizontally, federation uses multiple independent Namenodes/namespaces. The Namenodes are federated; the Namenodes are independent and do not require coordination with each other. The Datanodes are used as common storage for blocks by all the Namenodes. Each Datanode registers with all the Namenodes in the cluster. Datanodes send periodic heartbeats and block reports. They also handle commands from the Namenodes.
Users may use ViewFs to create personalized namespace views. ViewFs is analogous to client side mount tables in some Unix/Linux systems.
More on federation here.
Created 08-04-2016 08:38 PM
I had looked at this earlier, and while compelling, doesn't quite meet our needs. Thanks for the info!
Created 08-04-2016 06:39 PM
This feature is currently not available but will be shipped with Hadoop 3.0. You will be able to have up to 5 name nodes. Please see the following JIRA.
Created 08-04-2016 08:37 PM
This was exactly what I was looking for. Thanks for the heads up!