Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

AMBARI + HDFS is 100%

avatar

42761-capture.png

from ambari dashboard we can see that HDFS is 100%

I start with

hadoop fs -du -h / | grep spark2-history
60.9 G   /spark2-history

* other way to see who take space is from - https://community.hortonworks.com/articles/16846/how-to-identify-what-is-consuming-space-in-hdfs.htm...

is it mean that spark-history take from HDFS 60.9G ?

if not how the know what need to clean from the HDFS ?

we have 5 workers machine

and each worker have the following:

42762-capture.png

Michael-Bronson
1 ACCEPTED SOLUTION

avatar

from the article - How to identify what is consuming space in HDFS

* ( link https://community.hortonworks.com/articles/16846/how-to-identify-what-is-consuming-space-in-hdfs.htm... ) ,

by running the script ( from the article ) , we can see who take the most space

so in our case - spark-history take the most space , and we deleted the logs/files from Ambari-GUI

Michael-Bronson

View solution in original post

1 REPLY 1

avatar

from the article - How to identify what is consuming space in HDFS

* ( link https://community.hortonworks.com/articles/16846/how-to-identify-what-is-consuming-space-in-hdfs.htm... ) ,

by running the script ( from the article ) , we can see who take the most space

so in our case - spark-history take the most space , and we deleted the logs/files from Ambari-GUI

Michael-Bronson