Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

File distribution on HDFS

SOLVED Go to solution

File distribution on HDFS

Rising Star

Hi dear expert!

i'm wondering is there any way to check file distribution amond nodes in HDFS?

 

some way that allow to check on which nodes place some particular file of dirrectory?

 

thanks!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: File distribution on HDFS

Master Guru
If by 'hard to analyse' you mean to parse/process it, you can consider
using the Java API to fetch block location info too:
http://archive.cloudera.com/cdh5/cdh/5/hadoop/api/org/apache/hadoop/fs/FileSystem.html#getFileBlockL...

3 REPLIES 3

Re: File distribution on HDFS

Master Guru
You can check the file on NN Web UI's File Browser (shows all block IDs and
their locations), or run 'hdfs fsck /path/to/file -files -blocks -locations'

Re: File distribution on HDFS

Rising Star
Thanks! Hdfs fsck will works, but it's hard to analyze in case of big file. maybe it is other way to get aggregate values?

Highlighted

Re: File distribution on HDFS

Master Guru
If by 'hard to analyse' you mean to parse/process it, you can consider
using the Java API to fetch block location info too:
http://archive.cloudera.com/cdh5/cdh/5/hadoop/api/org/apache/hadoop/fs/FileSystem.html#getFileBlockL...