question Re: List out the Metadata attributes in Hadoop in Archives of Support Questions (Read Only)

List out the Metadata attributes in Hadoop

gsrao_cse — Mon, 18 Jul 2016 19:49:28 GMT

1. List out the Metadata attributes in Hadoop

2. Can we see the block level metadata file? if Yes, how can we see that file

Re: List out the Metadata attributes in Hadoop

mjohnson — Mon, 18 Jul 2016 21:14:48 GMT

1.) You can access the Metadata attributes in Hadoop using the following rest command: curl -i "http://server1:50070/webhdfs/v1/tmp?user.name=hdfs&op=GETFILESTATUS" where 'server1' is the setting for 'dfs.namenode.http-address' and 'tmp' is the directory you wanted to get the Metadata attributes for

{"FileStatus":{"accessTime":0,"blockSize":0,"childrenNum":6,"fileId":16386,"group":"hdfs","length":0,"modificationTime":1468810999066,"owner":"hdfs","pathSuffix":"","permission":"777","replication":0,"storagePolicy":0,"type":"DIRECTORY"}}[root@server1 ~]#

A link to more information about the WebHDFS REST API is available at: https://hadoop.apache.org/docs/r1.0.4/webhdfs.html#blocksize

2.) You can also list out the block information for a directory with the command line: 'hdfs fsck / -files -blocks -locations'

Re: List out the Metadata attributes in Hadoop

ArpitAgarwal — Tue, 19 Jul 2016 05:27:49 GMT

Note that 'hdfs fsck / -files -blocks -locations' is a workaround. It can be slow on large clusters. There is no efficient way to query the block locations of a file.