question Prevent HDFS from creating data directories in Archives of Support Questions (Read Only)

Prevent HDFS from creating data directories

cstefan — Thu, 18 Oct 2018 16:08:29 GMT

Hello community,

I am using HDP 3.0 and I have configured HDFS DataNode Directories as follow: /grid0/hadoop/hdfs/data,/grid1/hadoop/hdfs/data,/grid/hadoop/hdfs/data/

One of my hosts (lets call him hostA) has only one mount point (/grid) and the other host (hostB) has 2 mount points (/grid0 & /grid1)

HDFS automatically creates the directories grid0 and grid1 in "hostA" and grid in "hostB".

Is there any way to prevent HDFS from doing that?

Thanks in advance

asirna — Thu, 18 Oct 2018 16:25:23 GMT

That is the expected behaviour. If you want to achieve this, then you need to create separate configs for each host.

For ex: hostA will have only /grid in datanode dirs and hostB will have /grid0 and /grid1 in their datanode dirs.

You can do that using Ambari config groups

cstefan — Thu, 18 Oct 2018 17:29:54 GMT

Thank you so much for your help, this solved my problem.