Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How Clean HDFS in ambari?

Solved Go to solution
Highlighted

How Clean HDFS in ambari?

New Contributor

Hi, I've been starting with Ambari and Sandbox Horton works, but I have a problem.

I've been developing some MapReduce and Pig script to test the framework and I always do rm in the result output in order to not waste space disk, but with the time (1 month) the hdfs disk is fully in the ambari metrics and I don't recovery space spite of remove and remove files.

4749-sin-titu1lo.jpg

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: How Clean HDFS in ambari?

Super Collaborator

@BRivas garriv

It takes some manual effort but, first change the working directory to / (cd /).

then do du -h on every folder to find the disk usage. (Example: "du -h var" or "du -h usr").

Locate the folder which is taking up all disk space and try to delete irrelevant files from that folder.

View solution in original post

5 REPLIES 5

Re: How Clean HDFS in ambari?

Rising Star

If you do just rm you're actually moving your data to the Trash. In order to remove the data from HDFS and free space, when you do the rm you have to put the flag -skipTrash.

In order to delete the data from the trash, you can run:

hdfs dfs -expunge
Highlighted

Re: How Clean HDFS in ambari?

New Contributor

Hi, thanks for the response. I´ve try but dont work. The problem is that the disk is filling while it is on but not store results. I think it may be because that is not where logs can be store, that is, every day metrics mark a 1 % stored more despite I dont use the machine. And now with the time is 99%, yesterday have 98% like show the picture.

4748-sin-titulo.jpg

Highlighted

Re: How Clean HDFS in ambari?

Super Collaborator

@BRivas garriv

It takes some manual effort but, first change the working directory to / (cd /).

then do du -h on every folder to find the disk usage. (Example: "du -h var" or "du -h usr").

Locate the folder which is taking up all disk space and try to delete irrelevant files from that folder.

View solution in original post

Highlighted

Re: How Clean HDFS in ambari?

New Contributor

Thanks you so much, its works. The problem is that Atlas was comsuming 25GB of disk space in logs. You have any idea why this can happend? I never use Atlas app.

Highlighted

Re: How Clean HDFS in ambari?

Don't have an account?
Coming from Hortonworks? Activate your account here