Limit Kudu logs

[#1]

We have already 10+ GB logs for Kudu Masters and 30+ GB logs for Kudu Tablet Servers. How to limit the number of log files to keep? The Kudu Configuration Reference mentiones only a limit on a log file size (--max_log_size), but no limit for the number of log files.

[#2]

Based on my observation there is no limit for max number of files, but Kudu runs a compression on old logs. So you can easily remove all the logs *.gz

[#3]

This is not so good, because a user has to reason about old logs. With dozens of tablet servers, it is necessary to use some automated configuration management (Ansible or similar tool).

Usually it is a function of the logging framework itself (like Log4J).

[#4]

Let me know if you find a configuration for this.

[#5 · Accepted solution]

If I understand correctly, you are talking about the logs in the configured --log_dir. By default Kudu will keep 10 log files per severity level. There is a flag to change that value, but it's currently marked as "experimental". It has been in Kudu for some time, so not changing it to stable is probably a bit of an oversight. I opened an Apache Kudu jira (KUDU-2754) to change it to a stable config. 

In the mean time, you can use the --max_log_files configuration by unlocking experimental configurations via --unlock_experimental_flags.