Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

I Want to change hdfs default storge path

I Want to change hdfs default storge path

New Contributor

hi,everyone

I want to change hdfs default storage path use ambari,but find hbase datanode cant work

My question is :

  1. if i want to change hdfs data path? how to operate use Ambari?
  2. My opention is reight?

I change default DataNode path. defaut is . /hadoop/hdfs/data

now I change DataNode Path is :/hadoop/hdfs/data2

6549-001.png

I Restart datanode,namenode, I Find hbase cant work, cant get data

and I cant find path hadoop/hdfs/data2 in datanode node

6550-002.png

3 REPLIES 3

Re: I Want to change hdfs default storge path

@pan bocun

1. Did you tried restarting HDFS service after modifying the settings ?

2. Is the datanode up now ?

3. Can you login to datanode and check the path exist - ie

$ls -ltr /hadoop/hdfs/data2

Re: I Want to change hdfs default storge path

Expert Contributor

Please follow below below process.

  1. Stop the cluster.
  2. Go to the HDFS service.
  3. Select Configuration > View and Edit. (Before editing check is directory is created or not and owner should be "HDFS" )
  4. Click the DataNode Data Directory property.
  5. Remove the current directories and add new ones.
  6. Click Save Changes to commit the changes.
  7. Copy the contents under the previous directory to under the new directory.
  8. Start the cluster.

Your comments and accept are appreciated.

For more information follow below link.

https://community.hortonworks.com/articles/2308/how-to-move-or-change-the-hdfs-datanode-directorie.h...

Highlighted

Re: I Want to change hdfs default storge path

New Contributor

6637-009.png

thanks! I change Hdsf data path,but error occured, follower is errormessage ,why?

I Can't find which step is error?

ErrorMessage:

1.2016-08-13 07:03:07,498 WARN datanode.DataNode (DataNode.java:checkStorageLocations(2439)) - Invalid dfs.datanode.data.dir /mnt/data : EPERM: Operation not permitted at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method) at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:727) at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:502) at org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:140) at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:156) at org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker.checkDir(DataNode.java:2394) at org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations(DataNode.java:2436) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2418) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2310) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2357) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2538) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2562) 2016-08-13 07:03:07,513 ERROR datanode.DataNode (DataNode.java:secureMain(2545)) - Exception in secureMain java.io.IOException: All directories in dfs.datanode.data.dir are invalid: "/mnt/data/" at org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations(DataNode.java:2445) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2418) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2310) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2357) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2538) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2562) 2016-08-13 07:03:07,514 INFO util.ExitUtil (ExitUtil.java:terminate(124)) - Exiting with status 1 2016-08-13 07:03:07,516 INFO datanode.DataNode (LogAdapter.java:info(47)) - SHUTDOWN_MSG: