Reply
Highlighted
JKC
New Contributor
Posts: 1
Registered: ‎03-17-2019

List hdfs directories and sort based on directories' size

I have tried to list the hadoop directories in human readable format using the below command and it worked well :

hadoop fs -du -s -h <path_to_hadoop_folder>

Now I am trying to sort this output based on size descending or ascending but not able to do so

"-S" option sorts based on file size but it is not compatible with -du filesystem command. It works only with -ls

And I used the below command. It sorts based on file size but it is listing all the subdirectories as well.

hadoop fs -ls -S -h <path_to_hadoop_folder>

But what I want is to list the folders' size and sort them based on its size. It should not go into the folder and list all the sub-directories or files.

Cloudera Employee
Posts: 14
Registered: ‎08-16-2018

Re: List hdfs directories and sort based on directories' size

Hi, 

 

The second command mentioned by you (as below) lists only the directories and does not include its sub directories.

 

hadoop fs -ls -S -h <path_to_hadoop_folder>

 

Can you double check and share the result you see?

 

Thanks.