how do I change namenode and datanode dir for an existing NameNode HA Cluster

[#1]

Currently I have an existing cluster with Namenode HA -- I want to change the dfs.namenode.name.dir and dfs.datanode.data.dir to different from what is currently configured.

[#2]

Best way to change dfs.datanode.data.dir is to decommission the datanode, change the value and add it back. This would avoid directly access files and moving them. You could move the contents from directory to another (without decommission) but not recommended.

For namenode metadata directories, you can change this value and restart standby NN first. Once Standby NN comes back (and it will start using new directory), then restart active NN. It picks the new directory on restart as well. Once both NNs are up, you can clear the contents of the older metadata directories.

[#3]

I was doing this from Ambari, after changing the directories namenode would not come up and starting complaining about namenode dir is not formated. Should I format the dir at the namenode machines ?

[#4]

Are you just mounting volumes at a new location?

[#5]

@Arpit Agarwal yes correct

[#6 · Accepted solution]

@sgowda, thanks for confirming you just want to mount volumes at a new location. If you are just remounting then your existing HDFS metadata and data files will be present but under new Linux paths.

In that case decommissioning is not necessary. You just need to to update NameNode and DataNode configuration settings like dfs.namenode.name.dir, dfs.datanode.data.dir to point to the new locations. See this link for a full list of settings, not all may apply to you. Don't reformat the NN else you will lose all your data.

The simplest approach is:

1. Take a full cluster downtime and bring down all HDFS services.
2. Remount volumes at the new location on all affected nodes.
3. Update NN and DN configurations via Ambari to point to the new storage roots.
4. Restart services.

If you are not familiar with these settings I recommend learning more about HDFS first since its easy to lose data via administrative mistakes.