Welcome to the Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this topic

HDFS du and fsck command shows different storage value

avatar
Contributor

I drop partitions for manage table then check the size of the table. I run below command to check.

hdfs du and hdfs fsck command shows different value of storage size

du t_table show 7TB

du t_table for all the partition of the table shows 600gb

fsck t_table show 600gb

 

Why it shows different values

 

hdfs dfs -du -s -h /data/warehouse/developer.db/t_table/
7.0 T 21.1 T /data/warehouse/developer.db/t_table

hdfs dfs -du -s -h /data/warehouse/developer.db/t_table/*
31.0 G 93.0 G /data/warehouse/developer.db/t_table/day_string=2019-01-01
30.9 G 92.7 G /data/warehouse/developer.db/t_table/day_string=2019-01-02
31.0 G 92.9 G /data/warehouse/developer.db/t_table/day_string=2019-01-03
31.0 G 93.0 G /data/warehouse/developer.db/t_table/day_string=2019-01-04
31.0 G 92.9 G /data/warehouse/developer.db/t_table/day_string=2019-01-05
31.1 G 93.2 G /data/warehouse/developer.db/t_table/day_string=2019-01-06
31.0 G 93.1 G /data/warehouse/developer.db/t_table/day_string=2019-01-07
31.0 G 93.1 G /data/warehouse/developer.db/t_table/day_string=2019-01-08
31.2 G 93.6 G /data/warehouse/developer.db/t_table/day_string=2019-01-09
31.0 G 93.1 G /data/warehouse/developer.db/t_table/day_string=2019-01-10
31.1 G 93.2 G /data/warehouse/developer.db/t_table/day_string=2019-01-11
31.1 G 93.4 G /data/warehouse/developer.db/t_table/day_string=2019-01-12
31.1 G 93.3 G /data/warehouse/developer.db/t_table/day_string=2019-01-13
31.1 G 93.4 G /data/warehouse/developer.db/t_table/day_string=2019-01-14
31.2 G 93.5 G /data/warehouse/developer.db/t_table/day_string=2019-01-15
31.1 G 93.2 G /data/warehouse/developer.db/t_table/day_string=2019-01-16
31.2 G 93.5 G /data/warehouse/developer.db/t_table/day_string=2019-01-17
31.1 G 93.4 G /data/warehouse/developer.db/t_table/day_string=2019-01-18

hdfs fsck /data/warehouse/developer.db/t_table/
Total size: 600360486250 B
Total dirs: 19
Total files: 1660

Who agreed with this topic