how to limit the logs rotation under /var/log/ambari-metrics-collector

[#1]

how to limit the logs rotation under /var/log/ambari-metrics-collector

so logs will be only on the last month

under /var/log/ambari-metrics-collector we see logs from Jul , but we want only logs until month ( logs rotation no longer then month )

otal 7277584
-rw-r--r--. 1 ams  hadoop     13794 Jul 13  2017 collector-gc.log-201707131628
-rw-r--r--. 1 ams  hadoop      1430 Jul 13  2017 gc.log-201707131628
-rw-r--r--  1 ams  hadoop 126748287 Aug 14  2017 collector-gc.log-201707160637
-rw-r--r--  1 ams  hadoop 178518767 Aug 28  2017 gc.log-201707160637
-rw-r--r--. 1 ams  hadoop  83886155 Sep 14  2017 ambari-metrics-collector.log.30
-rw-r--r--  1 ams  hadoop  94522724 Sep 17  2017 collector-gc.log-201708280731
-rw-r--r--  1 ams  hadoop 128748829 Sep 17  2017 gc.log-201708280731
-rw-r--r--  1 ams  hadoop      2850 Sep 18  2017 collector-gc.log-201709181421
-rw-r--r--  1 ams  hadoop     60849 Sep 18  2017 gc.log-201709181421
-rw-r--r--  1 ams  hadoop      3517 Sep 18  2017 collector-gc.log-201709181424
-rw-r--r--  1 ams  hadoop     82562 Sep 18  2017 gc.log-201709181424
-rw-r--r--  1 ams  hadoop 117495363 Oct 19  2017 collector-gc.log-201709181508
-rw-r--r--  1 ams  hadoop 164381669 Oct 19  2017 gc.log-201709181508
-rw-r--r--  1 ams  hadoop     16634 Oct 19  2017 collector-gc.log-201710191738
-rw-r--r--  1 ams  hadoop     46149 Oct 19  2017 gc.log-201710191738
-rw-r--r--  1 ams  hadoop   7052909 Oct 23  2017 collector-gc.log-201710200633
-rw-r--r--  1 ams  hadoop  13763796 Oct 23  2017 gc.log-201710200633
-rw-r--r--  1 ams  hadoop    632288 Oct 23  2017 collector-gc.log-201710231021
-rw-r--r--  1 ams  hadoop   1048289 Oct 23  2017 gc.log-201710231021
-rw-r--r--  1 ams  hadoop   1927171 Oct 24  2017 collector-gc.log-201710231649
-rw-r--r--  1 ams  hadoop   3473874 Oct 24  2017 gc.log-201710231649
-rw-r--r--. 1 ams  hadoop 268435531 Oct 28  2017 hbase-ams-master-master02.sys654.com.log.5
-rw-r--r--  1 ams  hadoop  83886115 Nov  1  2017 ambari-metrics-collector.log.29
-rw-r--r--  1 ams  hadoop  79689213 Nov 15  2017 collector-gc.log-201710241352
-rw-r--r--  1 ams  hadoop 137289030 Nov 15  2017 gc.log-201710241352
-rw-r--r--  1 ams  hadoop   4736994 Nov 16  2017 collector-gc.log-201711151355
-rw-r--r--  1 ams  hadoop   8037548 Nov 16  2017 gc.log-201711151355
-rw-r--r--  1 ams  hadoop  15168598 Nov 19  2017 collector-gc.log-201711161246
-rw-r--r--  1 ams  hadoop  23919164 Nov 19  2017 gc.log-201711161246
-rw-r--r--  1 ams  hadoop  83720006 Dec  6 16:32 collector-gc.log-201711191241
-rw-r--r--  1 ams  hadoop 139759594 Dec  6 16:35 gc.log-201711191241
-rw-r--r--  1 ams  hadoop  83886082 Dec 17 10:03 ambari-metrics-collector.log.28
-rw-r--r--  1 ams  hadoop 110573406 Dec 25 16:10 collector-gc.log-201712061635
-rw-r--r--  1 ams  hadoop 184118442 Dec 25 16:13 gc.log-201712061635
-rw-r--r--  1 ams  hadoop      1440 Jan  1 04:38 gc.log-201712251613
-rw-r--r--  1 ams  hadoop  39360581 Jan  1 04:40 collector-gc.log-201712251613
-rw-r--r--  1 ams  hadoop      1437 Jan 10 00:03 gc.log-201801021532
-rw-r--r--  1 ams  hadoop  42005865 Jan 10 00:04 collector-gc.log-201801021532
-rw-r--r--  1 ams  hadoop      1439 Jan 10 01:04 gc.log-201801100004
-rw-r--r--  1 ams  hadoop    286342 Jan 10 01:05 collector-gc.log-201801100004
-rw-r--r--  1 ams  hadoop      3637 Jan 10 01:11 gc.log-201801100105
-rw-r--r--  1 ams  hadoop      3085 Jan 10 01:13 collector-gc.log-201801100105
-rw-r--r--  1 ams  hadoop      1437 Jan 10 01:16 gc.log-201801100113
-rw-r--r--  1 ams  hadoop      7230 Jan 10 01:18 collector-gc.log-201801100113
-rw-r--r--  1 ams  hadoop      3636 Jan 10 01:24 gc.log-201801100118
-rw-r--r--  1 ams  hadoop      3085 Jan 10 01:25 collector-gc.log-201801100118
-rw-r--r--  1 ams  hadoop      1438 Jan 10 01:35 gc.log-201801100126
-rw-r--r--  1 ams  hadoop     17357 Jan 10 01:36 collector-gc.log-201801100126

[#2]

Within Ambari, you can go to the config: Ambari UI --> Ambari Metrics --> Configs --> Advanced ams-log4j

There you should find this property, when being set to something like 31 in combination with a daily rotation you will only keep 1 month of logs:

log4j.appender.file.MaxBackupIndex=31

Or when you configure a monthly rotation, you set the number to 1.

It doesn't delete the existing older log files, you'll have to clean it manually.

[#3 · Accepted solution]

@Michael Bronson

The Logs (like collector-gc.log Or gc.log) produced by the JVM following are not controlled by the Log4j So there is no way to control them or their rotations using Log4j

-Xloggc:/var/log/ambari-metrics-collector/collector-gc.log-201806070708
-Xloggc:/var/log/ambari-metrics-collector/gc.log-201806070708

In order to control those GC log files you will need to rely on the OS utilities like "logrotate" or "cronjobs" to clear those logs.

.

JVM provides some options like "-XX:+UseGCLogFileRotation" however they are responsible for rotating the GC logs when the JVM process is keep running.

However You can however control the AMS component logs like HMaster logs or AMS collector logs using the log4j settings

like:

log4j.appender.file.MaxFileSize={{ams_log_max_backup_size}}MB
log4j.appender.file.MaxBackupIndex={{ams_log_number_of_backup_files}}

.