question Re: How to refresh HDFS rack topology changes without restarting namenodes in Support Questions

How to refresh HDFS rack topology changes without restarting namenodes

rda3mon — Sat, 07 Mar 2020 18:25:40 GMT

After adding new datanodes to the existing cluster and updating topology file. I am not able to refresh the namenode to pick up rack of the new node. It puts under `/default-rack`.

Anyway to refresh the rack topology changes without restarting namenodes?

Re: How to refresh HDFS rack topology changes without restarting namenodes

venkatsambath — Sun, 08 Mar 2020 07:26:15 GMT

@rda3mon Currently the feature you are looking for is not available, but for future version there is a jira in place - https://issues.apache.org/jira/browse/HDFS-11242

Re: How to refresh HDFS rack topology changes without restarting namenodes

rda3mon — Sun, 08 Mar 2020 13:37:47 GMT

Thanks for pointing me to the patch.

Re: How to refresh HDFS rack topology changes without restarting namenodes

rda3mon — Mon, 09 Mar 2020 01:58:36 GMT

For those who are looking for hdfs behind hbase solution. You can use Hbase favored node rack awareness, which depends on hdfs "net.topology.script.file.name" and more importantly refreshes dynamically. That worked for me perfectly.

Re: How to refresh HDFS rack topology changes without restarting namenodes

royliang — Thu, 25 Jul 2024 07:43:36 GMT

The correct process for adding a new node with the correct mapping is:

1. Update the topology file of the new DN.
2. Issue dfadmin-refreshnodes to update the new topology map in the NN.
3. Start DN only after (2) so that it picks up the correct mapping and the default mapping is not cached.