Support Questions
Find answers, ask questions, and share your expertise

HDFS + how to disable the "du -sk" verifcation on data node disks

we are using HDP cluster with *182* `data node` machines

 

*HDP version - 2.6.4
Ambari version 2.6.1*

 

we notes the following behavior on the `data nodes` machines ( its happens on all `data-node` machines and on all disks )

 

when we perform the command as above example:

 

ps -eo s,user,cmd | grep ^[RD]
D hdfs du -sk /grid/sdj/hadoop/hdfs/data/current/BP-1018134753-10.3.6.170-1530088122990
D hdfs du -sk /grid/sdm/hadoop/hdfs/data/current/BP-1018134753-10.3.6.170-1530088122990
R root ps -eo s,user,cmd

 

Note - each disk in the `data node` is `5.4 T Bytes`

 

we can see that `HDFS` is running the "du -sk" on the `data node` disks

we don't so like this , because the meaning of that is consuming high load CPU Avrg and sometimes even bad performance

 

we are understand that `HDFS` need to run the "`du -sk`" in order to verify the disks space , **but** on other hand its cost - high CPU load avrg and sometimes even poor performance


so - is it possible to tell `HDFS` in some way to disable this verification ?

 

Michael-Bronson
0 REPLIES 0
; ;