Created on 03-07-2020 09:06 AM - last edited on 03-07-2020 10:25 AM by ask_bill_brooks
After adding new datanodes to the existing cluster and updating topology file. I am not able to refresh the namenode to pick up rack of the new node. It puts under `/default-rack`.
Anyway to refresh the rack topology changes without restarting namenodes?
Created on 03-07-2020 11:25 PM - edited 03-07-2020 11:26 PM
@rda3mon Currently the feature you are looking for is not available, but for future version there is a jira in place - https://issues.apache.org/jira/browse/HDFS-11242
Created on 03-07-2020 11:25 PM - edited 03-07-2020 11:26 PM
@rda3mon Currently the feature you are looking for is not available, but for future version there is a jira in place - https://issues.apache.org/jira/browse/HDFS-11242
Created 03-08-2020 06:37 AM
Thanks for pointing me to the patch.
Created 03-08-2020 06:58 PM
For those who are looking for hdfs behind hbase solution. You can use Hbase favored node rack awareness, which depends on hdfs "net.topology.script.file.name" and more importantly refreshes dynamically. That worked for me perfectly.
Created 07-25-2024 12:43 AM
The correct process for adding a new node with the correct mapping is:
1. Update the topology file of the new DN.
2. Issue dfadmin-refreshnodes to update the new topology map in the NN.
3. Start DN only after (2) so that it picks up the correct mapping and the default mapping is not cached.