Created 06-04-2018 08:13 PM
how to limit the logs rotation under /var/log/ambari-metrics-collector
so logs will be only on the last month
under /var/log/ambari-metrics-collector we see logs from Jul , but we want only logs until month ( logs rotation no longer then month )
otal 7277584 -rw-r--r--. 1 ams hadoop 13794 Jul 13 2017 collector-gc.log-201707131628 -rw-r--r--. 1 ams hadoop 1430 Jul 13 2017 gc.log-201707131628 -rw-r--r-- 1 ams hadoop 126748287 Aug 14 2017 collector-gc.log-201707160637 -rw-r--r-- 1 ams hadoop 178518767 Aug 28 2017 gc.log-201707160637 -rw-r--r--. 1 ams hadoop 83886155 Sep 14 2017 ambari-metrics-collector.log.30 -rw-r--r-- 1 ams hadoop 94522724 Sep 17 2017 collector-gc.log-201708280731 -rw-r--r-- 1 ams hadoop 128748829 Sep 17 2017 gc.log-201708280731 -rw-r--r-- 1 ams hadoop 2850 Sep 18 2017 collector-gc.log-201709181421 -rw-r--r-- 1 ams hadoop 60849 Sep 18 2017 gc.log-201709181421 -rw-r--r-- 1 ams hadoop 3517 Sep 18 2017 collector-gc.log-201709181424 -rw-r--r-- 1 ams hadoop 82562 Sep 18 2017 gc.log-201709181424 -rw-r--r-- 1 ams hadoop 117495363 Oct 19 2017 collector-gc.log-201709181508 -rw-r--r-- 1 ams hadoop 164381669 Oct 19 2017 gc.log-201709181508 -rw-r--r-- 1 ams hadoop 16634 Oct 19 2017 collector-gc.log-201710191738 -rw-r--r-- 1 ams hadoop 46149 Oct 19 2017 gc.log-201710191738 -rw-r--r-- 1 ams hadoop 7052909 Oct 23 2017 collector-gc.log-201710200633 -rw-r--r-- 1 ams hadoop 13763796 Oct 23 2017 gc.log-201710200633 -rw-r--r-- 1 ams hadoop 632288 Oct 23 2017 collector-gc.log-201710231021 -rw-r--r-- 1 ams hadoop 1048289 Oct 23 2017 gc.log-201710231021 -rw-r--r-- 1 ams hadoop 1927171 Oct 24 2017 collector-gc.log-201710231649 -rw-r--r-- 1 ams hadoop 3473874 Oct 24 2017 gc.log-201710231649 -rw-r--r--. 1 ams hadoop 268435531 Oct 28 2017 hbase-ams-master-master02.sys654.com.log.5 -rw-r--r-- 1 ams hadoop 83886115 Nov 1 2017 ambari-metrics-collector.log.29 -rw-r--r-- 1 ams hadoop 79689213 Nov 15 2017 collector-gc.log-201710241352 -rw-r--r-- 1 ams hadoop 137289030 Nov 15 2017 gc.log-201710241352 -rw-r--r-- 1 ams hadoop 4736994 Nov 16 2017 collector-gc.log-201711151355 -rw-r--r-- 1 ams hadoop 8037548 Nov 16 2017 gc.log-201711151355 -rw-r--r-- 1 ams hadoop 15168598 Nov 19 2017 collector-gc.log-201711161246 -rw-r--r-- 1 ams hadoop 23919164 Nov 19 2017 gc.log-201711161246 -rw-r--r-- 1 ams hadoop 83720006 Dec 6 16:32 collector-gc.log-201711191241 -rw-r--r-- 1 ams hadoop 139759594 Dec 6 16:35 gc.log-201711191241 -rw-r--r-- 1 ams hadoop 83886082 Dec 17 10:03 ambari-metrics-collector.log.28 -rw-r--r-- 1 ams hadoop 110573406 Dec 25 16:10 collector-gc.log-201712061635 -rw-r--r-- 1 ams hadoop 184118442 Dec 25 16:13 gc.log-201712061635 -rw-r--r-- 1 ams hadoop 1440 Jan 1 04:38 gc.log-201712251613 -rw-r--r-- 1 ams hadoop 39360581 Jan 1 04:40 collector-gc.log-201712251613 -rw-r--r-- 1 ams hadoop 1437 Jan 10 00:03 gc.log-201801021532 -rw-r--r-- 1 ams hadoop 42005865 Jan 10 00:04 collector-gc.log-201801021532 -rw-r--r-- 1 ams hadoop 1439 Jan 10 01:04 gc.log-201801100004 -rw-r--r-- 1 ams hadoop 286342 Jan 10 01:05 collector-gc.log-201801100004 -rw-r--r-- 1 ams hadoop 3637 Jan 10 01:11 gc.log-201801100105 -rw-r--r-- 1 ams hadoop 3085 Jan 10 01:13 collector-gc.log-201801100105 -rw-r--r-- 1 ams hadoop 1437 Jan 10 01:16 gc.log-201801100113 -rw-r--r-- 1 ams hadoop 7230 Jan 10 01:18 collector-gc.log-201801100113 -rw-r--r-- 1 ams hadoop 3636 Jan 10 01:24 gc.log-201801100118 -rw-r--r-- 1 ams hadoop 3085 Jan 10 01:25 collector-gc.log-201801100118 -rw-r--r-- 1 ams hadoop 1438 Jan 10 01:35 gc.log-201801100126 -rw-r--r-- 1 ams hadoop 17357 Jan 10 01:36 collector-gc.log-201801100126
Created 06-07-2018 07:42 AM
The Logs (like collector-gc.log Or gc.log) produced by the JVM following are not controlled by the Log4j So there is no way to control them or their rotations using Log4j
-Xloggc:/var/log/ambari-metrics-collector/collector-gc.log-201806070708 -Xloggc:/var/log/ambari-metrics-collector/gc.log-201806070708
In order to control those GC log files you will need to rely on the OS utilities like "logrotate" or "cronjobs" to clear those logs.
.
JVM provides some options like "-XX:+UseGCLogFileRotation" however they are responsible for rotating the GC logs when the JVM process is keep running.
However You can however control the AMS component logs like HMaster logs or AMS collector logs using the log4j settings
like:
log4j.appender.file.MaxFileSize={{ams_log_max_backup_size}}MB log4j.appender.file.MaxBackupIndex={{ams_log_number_of_backup_files}}
.
Created 06-07-2018 07:18 AM
Within Ambari, you can go to the config: Ambari UI --> Ambari Metrics --> Configs --> Advanced ams-log4j
There you should find this property, when being set to something like 31 in combination with a daily rotation you will only keep 1 month of logs:
log4j.appender.file.MaxBackupIndex=31
Or when you configure a monthly rotation, you set the number to 1.
It doesn't delete the existing older log files, you'll have to clean it manually.
Created 06-07-2018 07:42 AM
The Logs (like collector-gc.log Or gc.log) produced by the JVM following are not controlled by the Log4j So there is no way to control them or their rotations using Log4j
-Xloggc:/var/log/ambari-metrics-collector/collector-gc.log-201806070708 -Xloggc:/var/log/ambari-metrics-collector/gc.log-201806070708
In order to control those GC log files you will need to rely on the OS utilities like "logrotate" or "cronjobs" to clear those logs.
.
JVM provides some options like "-XX:+UseGCLogFileRotation" however they are responsible for rotating the GC logs when the JVM process is keep running.
However You can however control the AMS component logs like HMaster logs or AMS collector logs using the log4j settings
like:
log4j.appender.file.MaxFileSize={{ams_log_max_backup_size}}MB log4j.appender.file.MaxBackupIndex={{ams_log_number_of_backup_files}}
.