Support Questions
Find answers, ask questions, and share your expertise

HDFS + how to disable the "du -sk" verifcation on data node disks

we are using HDP cluster with *182* `data node` machines


*HDP version - 2.6.4
Ambari version 2.6.1*


we notes the following behavior on the `data nodes` machines ( its happens on all `data-node` machines and on all disks )


when we perform the command as above example:


ps -eo s,user,cmd | grep ^[RD]
D hdfs du -sk /grid/sdj/hadoop/hdfs/data/current/BP-1018134753-
D hdfs du -sk /grid/sdm/hadoop/hdfs/data/current/BP-1018134753-
R root ps -eo s,user,cmd


Note - each disk in the `data node` is `5.4 T Bytes`


we can see that `HDFS` is running the "du -sk" on the `data node` disks

we don't so like this , because the meaning of that is consuming high load CPU Avrg and sometimes even bad performance


we are understand that `HDFS` need to run the "`du -sk`" in order to verify the disks space , **but** on other hand its cost - high CPU load avrg and sometimes even poor performance

so - is it possible to tell `HDFS` in some way to disable this verification ?


; ;