navigate to hdfs through directories

[#1]

How to access hdfs in the system? I understand we use hadoop commands for that but what if I actually want to see what directories it is physically stored in?

[#2 · Accepted solution]

@sameer lail

take a look at your hdfs-defaults.xml and look at the directory setting for dfs.data.dir. this is where you hdfs files are stored. You can also view this setting on ambari under hdfs tab under configuration.

[#3]

@sameer lail I do want to inform you that hdfs is not a posix file system. data is stored in blocks and then split onto the data nodes. The name node has information about all the files, all the data blocks which make up the file. So when you use hadoop fs to do file level actions.

[#4]

@sameer lail

Go to Ambari-> HDFS and search for dfs.datanode.data.dir

then go to your Linux server and navigate to respective directories to see the data.

Ex:

[root@adcp DATA]# pwd

/DATA

[root@adcpDATA]#ls -lrt

drwxr-xr-x. 3 root root 4096 Dec 15 2014 sdd1

drwxr-xr-x. 3 root root 4096 Dec 15 2014 sdc1

drwxr-xr-x. 3 root root 4096 Dec 15 2014 sdf1

drwxr-xr-x. 3 root root 4096 Dec 15 2014 sde1

drwxr-xr-x. 3 root root 4096 Dec 15 2014 sdg1