Support Questions
Find answers, ask questions, and share your expertise

Hadoop : HDFS Cluster running out of space even though space is available

Explorer

We have 4 datanode HDFS cluster ...there is large amount of space available on each data node of about 98gb ...but when i look at the datanode information .. it's only using about 10gb and running out of space ...

11501-datanode.png

How can we make it use all the 98gb and not run out of space as indicated in image

this is the disk space configuration

11502-diskspace.png

this is the hdfs-site.xml on name node

<property>
        <name>dfs.name.dir</name>
        <value>/test/hadoop/hadoopinfra/hdfs/namenode</value>
 </property>

this is the hdfs-site.xml under data node

<property>
        <name>dfs.data.dir</name>
        <value>/test/hadoop/hadoopinfra/hdfs/datanode</value>
 </property>

Eventhough /test has 98GB and hdfs is configured to use it it's not using it

Am I missing anything while doing the configuration changes? And how can we make sure 98GB is used?

7 REPLIES 7

Rising Star

check if the directory exists. If not, create it.

df -h /test/hadoop/hadoopinfra/hdfs/datanode

Explorer

yes it exists ..

Expert Contributor

@Aditya Mamidala

Can you run the following command on your Datanodes:

df -h /test/hadoop/hadoopinfra/hdfs/datanode

Explorer

i ran this is the output

11503-datanodetest.png

Expert Contributor

@Aditya Mamidala

Property dfs.data.dir & dfs.name.dir are deprecated. Please use the following properties.

dfs.datanode.data.dir
dfs.namenode.name.dir

https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-common/DeprecatedProperties.html

Explorer

tried changing it still no effect

Contributor

Did you solve this?