Support Questions
Find answers, ask questions, and share your expertise

Limit Kudu logs

Solved Go to solution

Limit Kudu logs

Explorer

We have already 10+ GB logs for Kudu Masters and 30+ GB logs for Kudu Tablet Servers. How to limit the number of log files to keep? The Kudu Configuration Reference mentiones only a limit on a log file size (--max_log_size), but no limit for the number of log files.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Limit Kudu logs

Cloudera Employee

If I understand correctly, you are talking about the logs in the configured --log_dir. By default Kudu will keep 10 log files per severity level. There is a flag to change that value, but it's currently marked as "experimental". It has been in Kudu for some time, so not changing it to stable is probably a bit of an oversight. I opened an Apache Kudu jira (KUDU-2754) to change it to a stable config. 

 

In the mean time, you can use the --max_log_files configuration by unlocking experimental configurations via --unlock_experimental_flags.

View solution in original post

4 REPLIES 4
Highlighted

Re: Limit Kudu logs

Master Collaborator
Based on my observation there is no limit for max number of files, but Kudu runs a compression on old logs. So you can easily remove all the logs *.gz
Highlighted

Re: Limit Kudu logs

Explorer

This is not so good, because a user has to reason about old logs. With dozens of tablet servers, it is necessary to use some automated configuration management (Ansible or similar tool).

Usually it is a function of the logging framework itself (like Log4J).

Highlighted

Re: Limit Kudu logs

Master Collaborator
Let me know if you find a configuration for this.
Highlighted

Re: Limit Kudu logs

Cloudera Employee

If I understand correctly, you are talking about the logs in the configured --log_dir. By default Kudu will keep 10 log files per severity level. There is a flag to change that value, but it's currently marked as "experimental". It has been in Kudu for some time, so not changing it to stable is probably a bit of an oversight. I opened an Apache Kudu jira (KUDU-2754) to change it to a stable config. 

 

In the mean time, you can use the --max_log_files configuration by unlocking experimental configurations via --unlock_experimental_flags.

View solution in original post