https://community.cloudera.com/t5/Support-Questions/How-to-increase-DFS-space-on-existing-cluster/m-p/187845#M149946 <P><A href="https://community.hortonworks.com/users/18739/sysadmin.html">@sysadmin CreditVidya</A> There are several approaches I can think of might help:</P><P>1. It appears MR intermediate data is not being purged properly by Hadoop itself, you can manually delete files/folders configured in mapreduce.cluster.local.dir after MR jobs are completed, say files/folders older than 3 days. You can probably create a cron job for that purpose.</P><P>2. Make sure to implement cleanup() method in each mapper/reducer class, which will clean up local resources, and aggregates before the task exists.</P><P>3. Run hdfs balancer regularly, normally weekly or bi-weekly, that way you won't have too much more hdfs data stored on some nodes comparing to the others, as MR jobs always try to use the local copy of the data first, and always keep an eye on 'disk usage' for each host in Ambari.</P><P>Hope that helps.</P> Wed, 19 Jul 2017 22:00:58 GMT dsun 2017-07-19T22:00:58Z