Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS_FREE_SPACE_REMAINING

HDFS_FREE_SPACE_REMAINING

New Contributor

Hi,

 

If a data node/all data nodes is a cluster has the above warning and the warning has gone to critical for less than 10% space, would cloudera manager then prevent any futher processing/applications from running on the cluster/datanodes?

 

best wishes

 

AM

3 REPLIES 3

Re: HDFS_FREE_SPACE_REMAINING

Cloudera Employee

Hi Adam88morris,

 

Cloudera Manager won't prevent running anything, but nevertheless applications can fail due to lack of free space. It's best to add new DataNodes to the cluster if you can.

 

Regards,

 

Re: HDFS_FREE_SPACE_REMAINING

New Contributor

Hi,

 

thanks for your reply. 

Do the applications use the critical limit as a marker for when they should not run?

we have created some space and the applications have began to run again,

 

thanks

AM

 

 

Highlighted

Re: HDFS_FREE_SPACE_REMAINING

Expert Contributor

No, no application that I know of interfaces with CM to influence it's behavior. however you can get some really weird behavior from any application that uses the "non-hdfs" space on the cluster if any one node fills up and you have not reserved any space for non-hdfs using 

dfs.datanode.du.reserved

Map/Reduce, and by extension, hive and any other program that uses map/reduce requires non-hdfs space for intermediate data. 

Don't have an account?
Coming from Hortonworks? Activate your account here