Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Limit Kudu logs

Solved Go to solution
Highlighted

Limit Kudu logs

Explorer

We have already 10+ GB logs for Kudu Masters and 30+ GB logs for Kudu Tablet Servers. How to limit the number of log files to keep? The Kudu Configuration Reference mentiones only a limit on a log file size (--max_log_size), but no limit for the number of log files.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Limit Kudu logs

Cloudera Employee

If I understand correctly, you are talking about the logs in the configured --log_dir. By default Kudu will keep 10 log files per severity level. There is a flag to change that value, but it's currently marked as "experimental". It has been in Kudu for some time, so not changing it to stable is probably a bit of an oversight. I opened an Apache Kudu jira (KUDU-2754) to change it to a stable config. 

 

In the mean time, you can use the --max_log_files configuration by unlocking experimental configurations via --unlock_experimental_flags.

4 REPLIES 4

Re: Limit Kudu logs

Master Collaborator
Based on my observation there is no limit for max number of files, but Kudu runs a compression on old logs. So you can easily remove all the logs *.gz

Re: Limit Kudu logs

Explorer

This is not so good, because a user has to reason about old logs. With dozens of tablet servers, it is necessary to use some automated configuration management (Ansible or similar tool).

Usually it is a function of the logging framework itself (like Log4J).

Re: Limit Kudu logs

Master Collaborator
Let me know if you find a configuration for this.

Re: Limit Kudu logs

Cloudera Employee

If I understand correctly, you are talking about the logs in the configured --log_dir. By default Kudu will keep 10 log files per severity level. There is a flag to change that value, but it's currently marked as "experimental". It has been in Kudu for some time, so not changing it to stable is probably a bit of an oversight. I opened an Apache Kudu jira (KUDU-2754) to change it to a stable config. 

 

In the mean time, you can use the --max_log_files configuration by unlocking experimental configurations via --unlock_experimental_flags.