Prevent HDFS from creating data directories

[#1]

Hello community,

I am using HDP 3.0 and I have configured HDFS DataNode Directories as follow: /grid0/hadoop/hdfs/data,/grid1/hadoop/hdfs/data,/grid/hadoop/hdfs/data/

One of my hosts (lets call him hostA) has only one mount point (/grid) and the other host (hostB) has 2 mount points (/grid0 & /grid1)

HDFS automatically creates the directories grid0 and grid1 in "hostA" and grid in "hostB".

Is there any way to prevent HDFS from doing that?

Thanks in advance

[#2 · Accepted solution]

@Christos Stefanopoulos,

That is the expected behaviour. If you want to achieve this, then you need to create separate configs for each host.

For ex: hostA will have only /grid in datanode dirs and hostB will have /grid0 and /grid1 in their datanode dirs.

You can do that using Ambari config groups

https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-operations/content/using_host_con...

[#3]

@Aditya Sirna

Thank you so much for your help, this solved my problem.