hi all
we have ambari cluster version 2.6.1 & HDP version 2.6.4
from the dashboard we can see that HDFS DISK Usage is almost 90%
but all data-node disk are around 90%
so why HDFS show 90% , while datanode disk are only 50%
/dev/sdc 20G 11G 8.7G 56% /data/sdc /dev/sde 20G 11G 8.7G 56% /data/sde /dev/sdd 20G 11G 9.0G 55% /data/sdd /dev/sdb 20G 8.9G 11G 46% /data/sdb
is it problem of fine-tune ? or else
we also performed re-balance from the ambari GUI but this isn't help
As the NameNode Report and UI (including ambari UI) shows that your DFS used is reaching almsot 87% to 90% hence it will be really good if you can increase the DFS capacity.
In order to understand in detail about the Non DFS Used = Configured Capacity - DFS Remaining - DFS Used
YOu can refer to the following article which aims at explaining the concepts of Configured Capacity, Present Capacity, DFS Used,DFS Remaining, Non DFS Used, in HDFS. The diagram below clearly explains these output space parameters assuming HDFS as a single disk.
https://community.hortonworks.com/articles/98936/details-of-the-output-hdfs-dfsadmin-report.html
.
The above is one of the best article to understand the DFS and Non-DFS calculations and remedy.
You add capacity by giving dfs.datanode.data.dir more mount points or directories. In Ambari that section of configs is I believe to the right depending the version of Ambari or in advanced section, the property is in hdfs-site.xml. the more new disk you provide through comma separated list the more capacity you will have. Preferably every machine should have same disk and mount point structure.
.
this is example what we have on the datanode machine , you can see that used are around 47-61% , and each disk is 20G size
