Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Any issues with truncating the tables in Ambari Metrics Service (AMS)?

Any issues with truncating the tables in Ambari Metrics Service (AMS)?

Contributor

I recently realized that the AMS service was writing its metric data to a set of HBase tables in a standalone HBase DB located here:

$ ls -l /var/lib/ambari-metrics-collector/hbase/data/default/
total 32
drwxr-xr-x 5 ams hadoop 4096 Aug  2 01:16 METRIC_AGGREGATE
drwxr-xr-x 5 ams hadoop 4096 Aug  2 01:17 METRIC_AGGREGATE_HOURLY
drwxr-xr-x 5 ams hadoop 4096 Aug  2 01:18 METRIC_RECORD
drwxr-xr-x 5 ams hadoop 4096 Aug  2 01:18 METRIC_RECORD_HOURLY
drwxr-xr-x 5 ams hadoop 4096 Aug  2 01:18 METRIC_RECORD_MINUTE
drwxr-xr-x 5 ams hadoop 4096 Sep 25  2015 SYSTEM.CATALOG
drwxr-xr-x 6 ams hadoop 4096 Sep 25  2015 SYSTEM.SEQUENCE
drwxr-xr-x 5 ams hadoop 4096 Sep 25  2015 SYSTEM.STATS

I also found out that you can tell the hbase CLI tool to use a specific configuration file like this:

$ hbase --config /etc/ams-hbase/conf shell

So when I ran this command I was happy to see the actual HBase tables, and a method to gain access to these tables via HBase shell:

hbase(main):001:0> list
TABLE
....
METRIC_AGGREGATE
METRIC_AGGREGATE_HOURLY
METRIC_RECORD
METRIC_RECORD_HOURLY
METRIC_RECORD_MINUTE
SYSTEM.CATALOG
SYSTEM.SEQUENCE
SYSTEM.STATS
8 row(s) in 0.7040 seconds

At this point I realized I could now create scripts to truncate and/or trim these AMS tables through a cron or any such method that I deemed necessary.

My Question

Is there any unforeseen issues in trimming AMS's tables in this manner or in truncating them?

2 REPLIES 2
Highlighted

Re: Any issues with truncating the tables in Ambari Metrics Service (AMS)?

Mentor

if this was the right approach, it would be clearly documented, I can't speak for unintended consequences but I wouldn't risk it. If goal is to clean up indefinitely, use steps here https://cwiki.apache.org/confluence/display/AMBARI/Cleaning+up+Ambari+Metrics+System+Data

If however the goal is to trim down the retention interval, I would tweak the purge interval for each table, https://cwiki.apache.org/confluence/display/AMBARI/Phoenix+Schema

Re: Any issues with truncating the tables in Ambari Metrics Service (AMS)?

Expert Contributor

Hi slm,

You can use the ttl options per table to truncate the tables accordingly. Please look for ams-site configs containing the word "ttl" . It can be done using the Ambari UI. The values are in seconds. Please restart AmS collector after the changes.

Thanks.

Don't have an account?
Coming from Hortonworks? Activate your account here