Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to purge old ambari metrics records

avatar


hi all


is it possible to purge old data from :

 /var/lib/ambari-metrics-collector/hbase/data/default ?


2.8G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/f051a7d687bdb4cea054e0332beb49ed/0
2.8G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/f051a7d687bdb4cea054e0332beb49ed
2.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/71b514c110ac2f143085b21c5d053a07/0
2.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/71b514c110ac2f143085b21c5d053a07
2.3G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/50d183833e81ba54490af42e1d8263d9/0
2.3G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/50d183833e81ba54490af42e1d8263d9
1.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/ae2a310c89be831741131a6b5bae95df/0
1.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/ae2a310c89be831741131a6b5bae95df
1.9G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/04c51a8735882c2b36f3a97aefe9354e/0
1.9G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/04c51a8735882c2b36f3a97aefe9354e
1.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/7b2f6cdb869a3156d81ab85f78ae9df6/0
1.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/7b2f6cdb869a3156d81ab85f78ae9df6
1.3G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/64ace9314b3c4ae4d58b7946d39a360f/0
1.3G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/64ace9314b3c4ae4d58b7946d39a360f
1.4G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/4a3c5f6bf1927cd11e34945f3e9cdd09/0
1.4G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/4a3c5f6bf1927cd11e34945f3e9cdd09
1.5G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/647d4583cbb3bcdb815bca0f4d529230/0
1.5G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/647d4583cbb3bcdb815bca0f4d529230
17G     /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD
2.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/53caf82013409e9d6f49379e3441db22/0
2.9G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/53caf82013409e9d6f49379e3441db22
2.5G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/685497e4b91e253debba8563be6dcb9f/0
3.1G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/685497e4b91e253debba8563be6dcb9f
1.2G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/52436005f0f9f211080d3ac3f5964796/0
1.2G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/52436005f0f9f211080d3ac3f5964796
2.4G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/3ba7fefae01d8345b74c7dd3e89a0488/0
2.4G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/3ba7fefae01d8345b74c7dd3e89a0488
2.3G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/ecea9d930a67cacd7d9b5080a8570b25/0
2.9G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/ecea9d930a67cacd7d9b5080a8570b25
2.2G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/16053dcb38d54a3b6da07e507788062b/0


/var is is the local disk and it is almost 100%




Michael-Bronson
4 REPLIES 4

avatar
Master Mentor

@Michael Bronson

Yes, you can filter by days in the below example I filtering all jpg files older than 30 days and running the delete thereafter

You can test by running first only

find /path/to/files/ -type f -name '*.jpg' -mtime +30 

then add the delete command

find /path/to/files/ -type f -name '*.jpg' -mtime +30 -exec rm {} \;

Hope that helps



avatar

I guess we need to stop the metrics collector service before we deleted the files - am I correct ?

second - is it safe to do this approach ( deleting the files ? )


Michael-Bronson

avatar

its not suggested to delete the Intermediate files of hbase directory directly. follow this : if you are ok with data erasing : https://cwiki.apache.org/confluence/display/AMBARI/Cleaning+up+Ambari+Metrics+System+Data

avatar

Hi @Michael Bronson ,

If you want to preserve the metrics data of AMS , please follow the steps here : https://community.hortonworks.com/questions/246159/ambari-metrics-change-the-path-of-hbaserootdir-be...


If you dont want to preserve the data of AMS and just want to clean it up : please refer to : https://cwiki.apache.org/confluence/display/AMBARI/Cleaning+up+Ambari+Metrics+System+Data

Refer to this blog if you are more interested in reading blogs : https://community.hortonworks.com/articles/144318/identifying-and-tackling-scale-problems-in-ambari....


Please accept answer if this Helps 🙂