Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Limit Kudu logs

avatar
Explorer

We have already 10+ GB logs for Kudu Masters and 30+ GB logs for Kudu Tablet Servers. How to limit the number of log files to keep? The Kudu Configuration Reference mentiones only a limit on a log file size (--max_log_size), but no limit for the number of log files.

1 ACCEPTED SOLUTION

avatar
Cloudera Employee

If I understand correctly, you are talking about the logs in the configured --log_dir. By default Kudu will keep 10 log files per severity level. There is a flag to change that value, but it's currently marked as "experimental". It has been in Kudu for some time, so not changing it to stable is probably a bit of an oversight. I opened an Apache Kudu jira (KUDU-2754) to change it to a stable config. 

 

In the mean time, you can use the --max_log_files configuration by unlocking experimental configurations via --unlock_experimental_flags.

View solution in original post

4 REPLIES 4

avatar
Based on my observation there is no limit for max number of files, but Kudu runs a compression on old logs. So you can easily remove all the logs *.gz

avatar
Explorer

This is not so good, because a user has to reason about old logs. With dozens of tablet servers, it is necessary to use some automated configuration management (Ansible or similar tool).

Usually it is a function of the logging framework itself (like Log4J).

avatar
Let me know if you find a configuration for this.

avatar
Cloudera Employee

If I understand correctly, you are talking about the logs in the configured --log_dir. By default Kudu will keep 10 log files per severity level. There is a flag to change that value, but it's currently marked as "experimental". It has been in Kudu for some time, so not changing it to stable is probably a bit of an oversight. I opened an Apache Kudu jira (KUDU-2754) to change it to a stable config. 

 

In the mean time, you can use the --max_log_files configuration by unlocking experimental configurations via --unlock_experimental_flags.