question Re: Impala Audit & Lineage Logs in Support Questions

Impala Audit & Lineage Logs

Amn_468 — Fri, 16 Sep 2022 14:41:47 GMT

Hello,

I need some assistance / guidance related to Impala Audit & Lineage Logs.

In our cluster I see Impala Audit Logs (enable_audit_event_log) Un-checked, yet on the node level we see logs getting generated.

[service1@myserver audit]# ls -ltrh total 215M -rw-r--r-- 1 impala impala 4.2M May 27 14:45 impala_audit_event_log_1.0-1622078304372 -rw-r--r-- 1 impala impala 3.9M May 27 15:34 impala_audit_event_log_1.0-1622083537006 -rw-r--r-- 1 impala impala 4.3M May 27 16:44 impala_audit_event_log_1.0-1622086468134 -rw-r--r-- 1 impala impala 4.3M May 27 17:18 impala_audit_event_log_1.0-1622090691805 -rw-r--r-- 1 impala impala 4.3M May 27 18:26 impala_audit_event_log_1.0-1622092726630 -rw-r--r-- 1 impala impala 4.2M May 27 20:07 impala_audit_event_log_1.0-1622096799288 -rw-r--r-- 1 impala impala 4.0M May 27 21:17 impala_audit_event_log_1.0-1622102850640 -rw-r--r-- 1 impala impala 3.7M May 28 01:36 impala_audit_event_log_1.0-1622107032783 -rw-r--r-- 1 impala impala 2.7M May 28 14:34 impala_audit_event_log_1.0-1622122581046 -rw-r--r-- 1 impala impala 4.7M May 28 18:17 impala_audit_event_log_1.0-1622169268919 -rw-r--r-- 1 impala impala 4.3M May 28 19:34 impala_audit_event_log_1.0-1622182634220 -rw-r--r-- 1 impala impala 4.3M May 28 20:30 impala_audit_event_log_1.0-1622187283341 -rw-r--r-- 1 impala impala 4.3M May 28 20:58 impala_audit_event_log_1.0-1622190632083 -rw-r--r-- 1 impala impala 4.4M May 28 21:25 impala_audit_event_log_1.0-1622192312991

Could anyone please share some light as to why this is happening? I was under the impression that if this is checked only then logs would get captured, kindly clarify

CM / CDH – 6.3.3

Thanks

Amn

Re: Impala Audit & Lineage Logs

salimhussain — Tue, 01 Jun 2021 05:16:45 GMT

This is correct, if Enable Impala Audit Event Generation is unchecked, it will disable Impala daemon Audit logging, See documentation [1]. I hope after setting this property, you would have restarted Impala daemon roles. If not, restart now and after restart you should not see new audit files being created but old files would still be there.

Can you check and confirm if these logs are generated on single Impala daemon hosts or on all Impala Daemon hosts even after it is disabled and Impala daemons restarted?

[1]

https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cn_admcfg_audit_log.html#configure_impala_daemon_log

Re: Impala Audit & Lineage Logs

VidyaSargur — Mon, 14 Jun 2021 05:46:25 GMT

Hi, @Amn_468 did you resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.

Re: Impala Audit & Lineage Logs

Amn_468 — Wed, 16 Jun 2021 07:22:49 GMT

Hi @salimhussain

Enable Impala Audit Event Generation property was already unchecked, for patching activity we had to bring down the cluster and start it back, so I hope that this counts as a restart of Impala too?

If it does, then we still have logs getting generated.

Thanks

Amn

Re: Impala Audit & Lineage Logs

salimhussain — Wed, 23 Jun 2021 07:17:19 GMT

Cluster restart also restarts Impala service. Could you please double check if following two properties are unchecked in CM > Impala > Configuration
Enable Audit Collection
Enable Impala Audit Event Generation

To be able to disable auditing for Impala service, you need to uncheck both of the above options. Once done make sure to restart Impala service. CM would show stale and restart icon as well.

Re: Impala Audit & Lineage Logs

VidyaSargur — Mon, 28 Jun 2021 06:34:48 GMT

@Amn_468, has @salimhussain's reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.

Re: Impala Audit & Lineage Logs

saini saab — Mon, 05 Jul 2021 07:53:10 GMT

Hello

I am suspecting that either of below are enabled in impala as these only generate audit logs.

Enable Audit Collection
Enable Impala Audit Event Generation

To be able to disable auditing for Impala service, you need to uncheck both of the above options. Once done make sure to restart Impala service. CM would show stale and restart icon as well.