How Clean HDFS in ambari?

[#1]

Hi, I've been starting with Ambari and Sandbox Horton works, but I have a problem.

I've been developing some MapReduce and Pig script to test the framework and I always do rm in the result output in order to not waste space disk, but with the time (1 month) the hdfs disk is fully in the ambari metrics and I don't recovery space spite of remove and remove files.

[#2]

If you do just rm you're actually moving your data to the Trash. In order to remove the data from HDFS and free space, when you do the rm you have to put the flag -skipTrash.

In order to delete the data from the trash, you can run:

hdfs dfs -expunge

[#3]

Hi, thanks for the response. I´ve try but dont work. The problem is that the disk is filling while it is on but not store results. I think it may be because that is not where logs can be store, that is, every day metrics mark a 1 % stored more despite I dont use the machine. And now with the time is 99%, yesterday have 98% like show the picture.

[#4 · Accepted solution]

@BRivas garriv

It takes some manual effort but, first change the working directory to / (cd /).

then do du -h on every folder to find the disk usage. (Example: "du -h var" or "du -h usr").

Locate the folder which is taking up all disk space and try to delete irrelevant files from that folder.

[#5]

Thanks you so much, its works. The problem is that Atlas was comsuming 25GB of disk space in logs. You have any idea why this can happend? I never use Atlas app.

[#6]

please check here:

https://community.hortonworks.com/questions/5148/atlas-generating-huge-log-files.html