Created 02-10-2018 08:14 AM
Hello,
i want to get details "Block-Locations" of a particular file say abc.csv(500mb)(Cluster: 1NM and 3DNs).
when i -put a file it is divided into blocks of default size 64MB and spread across Hadoop Cluster.
By using Web interface "http://namenode:50070" we can find out block location across cluster.
Also by using command : hadoop fsck <file-pat> -files -blocks -locations
But what i am trying to achieve is to get these information through JAVA API or WEB- API.
Please let me know the solution if any.
Any help will be appretiated.
Created 02-13-2018 07:23 PM
@Yog Prabhhu, you can get the file block information from WebHDFS REST API like
curl -i "http://<HOST>:<PORT>/webhdfs/v1/<FilePath>?op=GETFILEBLOCKLOCATIONS
The corresponding JAVA API is FileSystem.getFileBlockLocations:
public BlockLocation[] getFileBlockLocations(FileStatus file, long start, long len)
You will get an array of block locations like below:
[BlockLocation(offset: 0, length: BLOCK_SIZE,* hosts: {"host1:9866", "host2:9866, host3:9866"},...,]
Created 02-13-2018 07:23 PM
@Yog Prabhhu, you can get the file block information from WebHDFS REST API like
curl -i "http://<HOST>:<PORT>/webhdfs/v1/<FilePath>?op=GETFILEBLOCKLOCATIONS
The corresponding JAVA API is FileSystem.getFileBlockLocations:
public BlockLocation[] getFileBlockLocations(FileStatus file, long start, long len)
You will get an array of block locations like below:
[BlockLocation(offset: 0, length: BLOCK_SIZE,* hosts: {"host1:9866", "host2:9866, host3:9866"},...,]