Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

AMBARI + HDFS is 100%

avatar

42761-capture.png

from ambari dashboard we can see that HDFS is 100%

I start with

hadoop fs -du -h / | grep spark2-history
60.9 G   /spark2-history

* other way to see who take space is from - https://community.hortonworks.com/articles/16846/how-to-identify-what-is-consuming-space-in-hdfs.htm...

is it mean that spark-history take from HDFS 60.9G ?

if not how the know what need to clean from the HDFS ?

we have 5 workers machine

and each worker have the following:

42762-capture.png

Michael-Bronson
1 ACCEPTED SOLUTION

avatar

from the article - How to identify what is consuming space in HDFS

* ( link https://community.hortonworks.com/articles/16846/how-to-identify-what-is-consuming-space-in-hdfs.htm... ) ,

by running the script ( from the article ) , we can see who take the most space

so in our case - spark-history take the most space , and we deleted the logs/files from Ambari-GUI

Michael-Bronson

View solution in original post

1 REPLY 1

avatar

from the article - How to identify what is consuming space in HDFS

* ( link https://community.hortonworks.com/articles/16846/how-to-identify-what-is-consuming-space-in-hdfs.htm... ) ,

by running the script ( from the article ) , we can see who take the most space

so in our case - spark-history take the most space , and we deleted the logs/files from Ambari-GUI

Michael-Bronson