question Re: navigate to hdfs through directories in Archives of Support Questions (Read Only)

navigate to hdfs through directories

simran_k — Fri, 10 Jun 2016 12:23:28 GMT

How to access hdfs in the system? I understand we use hadoop commands for that but what if I actually want to see what directories it is physically stored in?

Re: navigate to hdfs through directories

sunile_manjee — Fri, 10 Jun 2016 12:23:29 GMT

@sameer lail

take a look at your hdfs-defaults.xml and look at the directory setting for dfs.data.dir. this is where you hdfs files are stored. You can also view this setting on ambari under hdfs tab under configuration.

Re: navigate to hdfs through directories

sunile_manjee — Fri, 10 Jun 2016 12:26:57 GMT

@sameer lail I do want to inform you that hdfs is not a posix file system. data is stored in blocks and then split onto the data nodes. The name node has information about all the files, all the data blocks which make up the file. So when you use hadoop fs to do file level actions.

Re: navigate to hdfs through directories

divakarreddy_a — Fri, 10 Jun 2016 12:36:21 GMT

@sameer lail

Go to Ambari-> HDFS and search for dfs.datanode.data.dir

then go to your Linux server and navigate to respective directories to see the data.

Ex:

[root@adcp DATA]# pwd

/DATA

[root@adcpDATA]#ls -lrt

drwxr-xr-x. 3 root root 4096 Dec 15 2014 sdd1

drwxr-xr-x. 3 root root 4096 Dec 15 2014 sdc1

drwxr-xr-x. 3 root root 4096 Dec 15 2014 sdf1

drwxr-xr-x. 3 root root 4096 Dec 15 2014 sde1

drwxr-xr-x. 3 root root 4096 Dec 15 2014 sdg1