question Re: what should be the value for dfs.datanode.data.dir in hdfs-site.xml in Archives of Support Questions (Read Only)

what should be the value for dfs.datanode.data.dir in hdfs-site.xml

Jacqueline — Thu, 19 Jan 2017 08:52:57 GMT

hi , I have

<property> <name>dfs.datanode.data.dir</name> <value>/hadoop/hdfs/data</value><final>true</final></property

When i cd to /hadoop/hdfs/data , i see below files

[hdfs@xxxxxxxxxx]$ pwd

/hadoop/hdfs/data

[hdfs@xxxxxxxxxxx]$ ls -ltr

total 8

-rw-r--r-- 1 hdfs hadoop 43 Jan 19 00:01 in_use.lock

drwxr-xr-x 5 hdfs hadoop 4096 Jan 19 00:01 current

My understand , was dfs.datanode.data.dir , should hold a series of disks values like ‘/u01/hadoop/data, /u02/hadoop/data’ …. where hdfs will store the data. In that case where will be current directory of datanode reside , which holds the VERSION file.

Re: what should be the value for dfs.datanode.data.dir in hdfs-site.xml

mpandit — Thu, 19 Jan 2017 11:57:35 GMT

All the HDFS meta directories explained here, http://hortonworks.com/blog/hdfs-metadata-directories-explained/ hope this helps.

Re: what should be the value for dfs.datanode.data.dir in hdfs-site.xml

rguruvannagari — Thu, 19 Jan 2017 13:37:37 GMT

@Jacqualin jasmin

dfs.datanode.data.dir can be any directory which is available on the datanode. It can be a directory where disk partitions are mounted like '/u01/hadoop/data, /u02/hadoop/data' which is in case if you have multiple disks partitions to be used for hdfs purpose.

Re: what should be the value for dfs.datanode.data.dir in hdfs-site.xml

hardikv_desai — Thu, 19 Jan 2017 18:54:27 GMT

@Jacqualin jasmin, you name any directory on your datanodes as the data directory. Make sure of the permissions and the value of the dfs.datanode.data.dir parameter in HDFS configurations.

e.g. /dev/vdb 1.5T 70M 1.4T 1% /volumes/disk1

dfs.datanode.data.dir=/volumes/disk1/hadoop/data/

after that you will find the VERSION file into the data directory once your HDFS detects the data directory.