Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Impala Deamon Logs

SOLVED Go to solution

Impala Deamon Logs

New Contributor

Hello All,

 

Could someone appraise me the significance of Impala Audit & Lineage Logs. We are facing space issue and would like  to know whether we can delete some of those old log files, also the default setting for Impala Daemon Maximum Lineage Log File Size & Impala Daemon Maximum Audit Log File Size is 5000 can this be lowered to 1000 etc.

 

Regards

Wert

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Impala Deamon Logs

Master Guru
As @Tomas79 explains, there will be no consequence whatsoever of making
that change (for your described problem) as these files are not deleted by
the writer (in the same way regular service log files are).

You'll need to delete older log files on your own, regardless of what you
specify the maximum file sizes to be for each rolled log. You can consider
using something like logrotate on Linux to automate this.
5 REPLIES 5

Re: Impala Deamon Logs

Master Collaborator
You can remove them, if you dont need them. Actually it should be part of your maintenamce, as at least what I know, there is no "auto rotation" or "log delete" on lineage logs.

Re: Impala Deamon Logs

Master Guru
To add on: If you will not require audits or lineage at all for your cluster, you can also choose to disable their creation:

Impala - Configuration - "Enable Impala Lineage Generation" (uncheck)
Impala - Configuration - "Enable Impala Audit Event Generation" (uncheck)

If you are using Navigator with Cloudera Enterprise, then these audits and lineage files should be sent automatically to the Navigator services. If they are not passing through, it may be an indicator of problem in the pipeline - please raise a support case if this is true.

Re: Impala Deamon Logs

New Contributor

Hi Harsh J / Tomas79,

 

Thank you for your responses, instead of disabling Audit & Lineage Log collection, I would be more inclined if can I reduce the default settings of 5000 to 1000, would there be any negative consequences?

 

Appreciate your insights / advice.

Re: Impala Deamon Logs

Master Guru
As @Tomas79 explains, there will be no consequence whatsoever of making
that change (for your described problem) as these files are not deleted by
the writer (in the same way regular service log files are).

You'll need to delete older log files on your own, regardless of what you
specify the maximum file sizes to be for each rolled log. You can consider
using something like logrotate on Linux to automate this.

Re: Impala Deamon Logs

New Contributor

Thank you all for your help