Skip to main content
kiranpune
Silver ⭐️⭐️
January 19, 2020
Solved

HDFS

I have removed files from HDFS by skipping trash but still it shows same disk useage

amk
amkAnswer
Gold ⭐️
January 19, 2020

How much data did you delete ? Did checkpoint happen after you deleted data ? Also please check if any snapshots are present ? HDFS CLI "du" output not only include normal files but also includes the files that have been deleted and exist in snapshots (which is true in terms of real resource consumption).

 

Please check the output using -x flag which excludes snapshot from calculation.

 

hdfs dfs -du -x -s -h /path

 

 

1 reply

amk
Gold ⭐️
amkAnswer
Gold ⭐️
January 19, 2020

How much data did you delete ? Did checkpoint happen after you deleted data ? Also please check if any snapshots are present ? HDFS CLI "du" output not only include normal files but also includes the files that have been deleted and exist in snapshots (which is true in terms of real resource consumption).

 

Please check the output using -x flag which excludes snapshot from calculation.

 

hdfs dfs -du -x -s -h /path

 

 

Translate