Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

How to purge old ambari metrics records


hi all


is it possible to purge old data from :

 /var/lib/ambari-metrics-collector/hbase/data/default ?


2.8G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/f051a7d687bdb4cea054e0332beb49ed/0
2.8G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/f051a7d687bdb4cea054e0332beb49ed
2.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/71b514c110ac2f143085b21c5d053a07/0
2.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/71b514c110ac2f143085b21c5d053a07
2.3G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/50d183833e81ba54490af42e1d8263d9/0
2.3G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/50d183833e81ba54490af42e1d8263d9
1.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/ae2a310c89be831741131a6b5bae95df/0
1.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/ae2a310c89be831741131a6b5bae95df
1.9G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/04c51a8735882c2b36f3a97aefe9354e/0
1.9G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/04c51a8735882c2b36f3a97aefe9354e
1.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/7b2f6cdb869a3156d81ab85f78ae9df6/0
1.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/7b2f6cdb869a3156d81ab85f78ae9df6
1.3G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/64ace9314b3c4ae4d58b7946d39a360f/0
1.3G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/64ace9314b3c4ae4d58b7946d39a360f
1.4G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/4a3c5f6bf1927cd11e34945f3e9cdd09/0
1.4G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/4a3c5f6bf1927cd11e34945f3e9cdd09
1.5G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/647d4583cbb3bcdb815bca0f4d529230/0
1.5G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/647d4583cbb3bcdb815bca0f4d529230
17G     /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD
2.6G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/53caf82013409e9d6f49379e3441db22/0
2.9G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/53caf82013409e9d6f49379e3441db22
2.5G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/685497e4b91e253debba8563be6dcb9f/0
3.1G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/685497e4b91e253debba8563be6dcb9f
1.2G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/52436005f0f9f211080d3ac3f5964796/0
1.2G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/52436005f0f9f211080d3ac3f5964796
2.4G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/3ba7fefae01d8345b74c7dd3e89a0488/0
2.4G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/3ba7fefae01d8345b74c7dd3e89a0488
2.3G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/ecea9d930a67cacd7d9b5080a8570b25/0
2.9G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/ecea9d930a67cacd7d9b5080a8570b25
2.2G    /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/16053dcb38d54a3b6da07e507788062b/0


/var is is the local disk and it is almost 100%




Michael-Bronson
4 REPLIES 4

Mentor

@Michael Bronson

Yes, you can filter by days in the below example I filtering all jpg files older than 30 days and running the delete thereafter

You can test by running first only

find /path/to/files/ -type f -name '*.jpg' -mtime +30 

then add the delete command

find /path/to/files/ -type f -name '*.jpg' -mtime +30 -exec rm {} \;

Hope that helps



I guess we need to stop the metrics collector service before we deleted the files - am I correct ?

second - is it safe to do this approach ( deleting the files ? )


Michael-Bronson

its not suggested to delete the Intermediate files of hbase directory directly. follow this : if you are ok with data erasing : https://cwiki.apache.org/confluence/display/AMBARI/Cleaning+up+Ambari+Metrics+System+Data

Hi @Michael Bronson ,

If you want to preserve the metrics data of AMS , please follow the steps here : https://community.hortonworks.com/questions/246159/ambari-metrics-change-the-path-of-hbaserootdir-be...


If you dont want to preserve the data of AMS and just want to clean it up : please refer to : https://cwiki.apache.org/confluence/display/AMBARI/Cleaning+up+Ambari+Metrics+System+Data

Refer to this blog if you are more interested in reading blogs : https://community.hortonworks.com/articles/144318/identifying-and-tackling-scale-problems-in-ambari....


Please accept answer if this Helps 🙂