Reply
New Contributor
Posts: 2
Registered: ‎02-06-2017

How we can add more namenode (slave cluster) to existing setup

I am new to Hadoop, and learning Hadoop.

 

My question is:

 

Let say i have one master (data node) and 3 slave (name nodes), Hadoop is configured and running perfectly with no problem.

 

After few months, if i want to configure another 2 name nodes, how that work.

 

My understanding is:

1 Make ready name node with linux OS

2 and there may be way to configure 2 namenodes in Hadoop (How i dont know yet)

3 After complition of task #2, hadoop will start using other 2 namenodes, means now hadoop is using 5 namenodes in total.

 

Thanks, V

Champion
Posts: 756
Registered: ‎05-16-2016

Re: How we can add more namenode (slave cluster) to existing setup

I will highlight the steps for configuring High availablity 
we have to modify  core-site.xml and  hdfs-site.xml 
Installation of  Journal nodes 
installation of ZooKeeper  nodes and ensemble
We need zookeeper failover controller 
we have to initialize the shared edits  directory
bootstraping the new namenodes.
 
There is no secondarynamenode the standby namenode will perform the checkpointing. 
 
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.ht...
 
hope this helps.
thanks 

Highlighted
Posts: 642
Topics: 3
Kudos: 115
Solutions: 67
Registered: ‎08-16-2016

Re: How we can add more namenode (slave cluster) to existing setup

There are two different setups that include multiple NNs.

Namenode High Availabily: A second Namenode is set up and configured along with either JournalNodes. The roles work together to provide a consistent state of the metadata tot he standby Namenode so that it can take over in the event that the first one fails. CM has a wizard to help set this up and it is well mature.

Namenode Federation: Many NNs can get installed and configured to function as one. Each NN is assigned a portion of the metadata. This allows you to scale beyond the memory capacity of a single server. This is not terrible complex but since it is a specific case there is no wizard. This should a lonely be used if you can't scale up on the NN any longer. I'd rather migrate to a bigger server than implement NN Federation.
Announcements