- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to refresh HDFS rack topology changes without restarting namenodes
- Labels:
-
Apache Hadoop
-
HDFS
Created on
‎03-07-2020
09:06 AM
- last edited on
‎03-07-2020
10:25 AM
by
ask_bill_brooks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
After adding new datanodes to the existing cluster and updating topology file. I am not able to refresh the namenode to pick up rack of the new node. It puts under `/default-rack`.
Anyway to refresh the rack topology changes without restarting namenodes?
Created on ‎03-07-2020 11:25 PM - edited ‎03-07-2020 11:26 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@rda3mon Currently the feature you are looking for is not available, but for future version there is a jira in place - https://issues.apache.org/jira/browse/HDFS-11242
Created on ‎03-07-2020 11:25 PM - edited ‎03-07-2020 11:26 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@rda3mon Currently the feature you are looking for is not available, but for future version there is a jira in place - https://issues.apache.org/jira/browse/HDFS-11242
Created ‎03-08-2020 06:37 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for pointing me to the patch.
Created ‎03-08-2020 06:58 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
For those who are looking for hdfs behind hbase solution. You can use Hbase favored node rack awareness, which depends on hdfs "net.topology.script.file.name" and more importantly refreshes dynamically. That worked for me perfectly.
Created ‎07-25-2024 12:43 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The correct process for adding a new node with the correct mapping is:
1. Update the topology file of the new DN.
2. Issue dfadmin-refreshnodes to update the new topology map in the NN.
3. Start DN only after (2) so that it picks up the correct mapping and the default mapping is not cached.
