Non DFS Utilization shows same post cleanup of local filesystem

ahshanmd — Thu, 12 Apr 2018 22:32:29 GMT

Initially when the environment was built there were around 327.33 GB out of total 1 TB Disk Capacity

Hence the HDFS dfsadmin report showed non-dfs usage as 327.33 GB

But after cleaning up of 300GB of on data from the Fileystem , dfsadmin report still show the non-dfs usage as 327.33 GB itself , while the reserved diskspace is 10GB

how could i get the Non-DFS utilisation refreshed post clean-up of local files on Linux Filesystem ?

hdfs dfsadmin -report

Name: 20.21.208.21:2004 (hpc123.xyz.com)
Hostname:hpc123.xyz.com
Rack: /Row7/Rack2
Decommission Status : Normal
Configured Capacity: 1154570731520 (1.05 TB)
DFS Used: 449671168 (428.84 MB)
Non DFS Used: 351465279488 (327.33 GB)
DFS Remaining: 802655780864 (747.53 GB)
DFS Used%: 0.04%
DFS Remaining%: 69.52%
Configured Cache Capacity: 4294967296 (4 GB)
Cache Used: 0 (0 B)
Cache Remaining: 4294967296 (4 GB)
Cache Used%: 0.00%
Cache Remaining%: 100.00%
Xceivers: 2

Last contact: Thu Apr 12 08:18:16 PDT 2018

Re: Non DFS Utilization shows same post cleanup of local filesystem

xyao — Sat, 14 Apr 2018 06:36:05 GMT

Have you clean up files under dfs.datanode.data.dirs that is not being written by HDFS for blocks? If not, the non-dfs used won't change.

Similar question has been answered here: https://community.hortonworks.com/questions/42122/hdfs-non-dfs-used.html.

question Non DFS Utilization shows same post cleanup of local filesystem in Archives of Support Questions (Read Only)

Non DFS Utilization shows same post cleanup of local filesystem

Re: Non DFS Utilization shows same post cleanup of local filesystem