Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

List hdfs directories and sort based on directories' size

Highlighted

List hdfs directories and sort based on directories' size

New Contributor

I have tried to list the hadoop directories in human readable format using the below command and it worked well :

hadoop fs -du -s -h <path_to_hadoop_folder>

Now I am trying to sort this output based on size descending or ascending but not able to do so

"-S" option sorts based on file size but it is not compatible with -du filesystem command. It works only with -ls

And I used the below command. It sorts based on file size but it is listing all the subdirectories as well.

hadoop fs -ls -S -h <path_to_hadoop_folder>

But what I want is to list the folders' size and sort them based on its size. It should not go into the folder and list all the sub-directories or files.

1 REPLY 1

Re: List hdfs directories and sort based on directories' size

Cloudera Employee

Hi, 

 

The second command mentioned by you (as below) lists only the directories and does not include its sub directories.

 

hadoop fs -ls -S -h <path_to_hadoop_folder>

 

Can you double check and share the result you see?

 

Thanks.