Member since
12-26-2016
15
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4983 | 08-19-2018 03:43 AM |
08-19-2018
03:43 AM
One is large environment with 20+ pb in size and data is completely different from other environment data, and reasons for different lakes are they both fall in different internal departments and data is also different and customers are also different, again depends on the data these cluster(s) servers located in different data centers and one is open for company wide enterprise network and others open for an internal network within enterprise network.
... View more
04-25-2017
09:22 PM
Hi @hdpadmin overlandpark Thanks for working with me over the Support portal. I will put the steps here for others reference: The Topology information is CACHED in NN and so, when ever there is change to Topology we need to restart the NN to clear it.
In our case Datanode is originally registered without rack (default-rack) and due to caching, NN will not allow it to join when its online.
So to fix this issue, we have to restart namenode, which will pick up the current updated rack information.
To avoid a NN restart, there is a workaround to add a new DataNode to a specific Rack without the need of restarting the NameNode: a. Add the New Node in Ambari without choosing 'DataNode' component
b. From Ambari Hosts tab, Select the new Node and add 'DataNode' component
c. Click 'Host Actions' -> 'Set Rack' and specify the required Rack Name
d. From the Ambari Hosts tab, go to the NameNode Host and Refresh the client configs by dropping down the Menu next to Clients.
This step will update /etc/hadoop/conf/topology_mappings.data on the NameNode with the new topology information for the DN.
e. Start the DataNode service on the new Node.
f. Once the Datanode is up, confirm its Rack topology information, by running the following command on any node with hdfs client:
# su - hdfs
# hdfs dfsadmin -report
In short, the Topology should be updated before a Datanode is started (at which point it tries to register with the NN)
... View more