question Re: Limit Kudu logs in Support Questions

Limit Kudu logs

arseniy — Fri, 16 Sep 2022 14:16:10 GMT

We have already 10+ GB logs for Kudu Masters and 30+ GB logs for Kudu Tablet Servers. How to limit the number of log files to keep? The Kudu Configuration Reference mentiones only a limit on a log file size (--max_log_size), but no limit for the number of log files.

Re: Limit Kudu logs

Tomas79 — Fri, 29 Mar 2019 08:00:18 GMT

Based on my observation there is no limit for max number of files, but Kudu runs a compression on old logs. So you can easily remove all the logs *.gz

Re: Limit Kudu logs

arseniy — Fri, 29 Mar 2019 11:22:30 GMT

This is not so good, because a user has to reason about old logs. With dozens of tablet servers, it is necessary to use some automated configuration management (Ansible or similar tool).

Usually it is a function of the logging framework itself (like Log4J).

Re: Limit Kudu logs

Tomas79 — Fri, 29 Mar 2019 12:07:44 GMT

Let me know if you find a configuration for this.

Re: Limit Kudu logs

Grant Henke — Fri, 29 Mar 2019 16:11:05 GMT

If I understand correctly, you are talking about the logs in the configured --log_dir. By default Kudu will keep 10 log files per severity level. There is a flag to change that value, but it's currently marked as "experimental". It has been in Kudu for some time, so not changing it to stable is probably a bit of an oversight. I opened an Apache Kudu jira (KUDU-2754) to change it to a stable config.

In the mean time, you can use the --max_log_files configuration by unlocking experimental configurations via --unlock_experimental_flags.