Impala Audit & Lineage Logs

[#1]

Hello,

I need some assistance / guidance related to Impala Audit & Lineage Logs.

In our cluster I  see Impala Audit Logs (enable_audit_event_log) Un-checked, yet on the node level we see logs getting generated.

[service1@myserver audit]# ls -ltrh
total 215M
-rw-r--r-- 1 impala impala 4.2M May 27 14:45 impala_audit_event_log_1.0-1622078304372
-rw-r--r-- 1 impala impala 3.9M May 27 15:34 impala_audit_event_log_1.0-1622083537006
-rw-r--r-- 1 impala impala 4.3M May 27 16:44 impala_audit_event_log_1.0-1622086468134
-rw-r--r-- 1 impala impala 4.3M May 27 17:18 impala_audit_event_log_1.0-1622090691805
-rw-r--r-- 1 impala impala 4.3M May 27 18:26 impala_audit_event_log_1.0-1622092726630
-rw-r--r-- 1 impala impala 4.2M May 27 20:07 impala_audit_event_log_1.0-1622096799288
-rw-r--r-- 1 impala impala 4.0M May 27 21:17 impala_audit_event_log_1.0-1622102850640
-rw-r--r-- 1 impala impala 3.7M May 28 01:36 impala_audit_event_log_1.0-1622107032783
-rw-r--r-- 1 impala impala 2.7M May 28 14:34 impala_audit_event_log_1.0-1622122581046
-rw-r--r-- 1 impala impala 4.7M May 28 18:17 impala_audit_event_log_1.0-1622169268919
-rw-r--r-- 1 impala impala 4.3M May 28 19:34 impala_audit_event_log_1.0-1622182634220
-rw-r--r-- 1 impala impala 4.3M May 28 20:30 impala_audit_event_log_1.0-1622187283341
-rw-r--r-- 1 impala impala 4.3M May 28 20:58 impala_audit_event_log_1.0-1622190632083
-rw-r--r-- 1 impala impala 4.4M May 28 21:25 impala_audit_event_log_1.0-1622192312991

Could anyone please share some light as to why this is happening? I was under the impression that if this is checked only then logs would get captured, kindly clarify

CM / CDH – 6.3.3

Thanks 

Amn

[#2 · Accepted solution]

Cluster restart also restarts Impala service. Could you please double check if following two properties are unchecked in CM > Impala > Configuration  
Enable Audit Collection
Enable Impala Audit Event Generation

To be able to disable auditing for Impala service, you need to uncheck both of the above options. Once done make sure to restart Impala service. CM would show stale and restart icon as well.  

[#3]

This is correct, if Enable Impala Audit Event Generation is unchecked, it will disable Impala daemon Audit logging, See documentation [1]. I hope after setting this property, you would have restarted Impala daemon roles. If not, restart now and after restart you should not see new audit files being created but old files would still be there. 

Can you check and confirm if these logs are generated on single Impala daemon hosts or on all Impala Daemon hosts even after it is disabled and Impala daemons restarted?

[1]

https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cn_admcfg_audit_log.html#configure_i...

[#4]

Hi, @Amn_468 did you resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

[#5]

Hi @salimhussain 

Enable Impala Audit Event Generation property was already unchecked, for patching activity we had to bring down the cluster and start it back, so I hope that this counts as a restart of Impala too?

If it does, then we still have logs getting generated.

Thanks 

Amn

[#6 · Accepted solution]

Cluster restart also restarts Impala service. Could you please double check if following two properties are unchecked in CM > Impala > Configuration  
Enable Audit Collection
Enable Impala Audit Event Generation

To be able to disable auditing for Impala service, you need to uncheck both of the above options. Once done make sure to restart Impala service. CM would show stale and restart icon as well.  

[#7]

@Amn_468, has @salimhussain's reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

[#8 · Accepted solution]

Hello 

I am suspecting that either of below are enabled in impala as these only generate audit logs.

Enable Audit Collection
Enable Impala Audit Event Generation

To be able to disable auditing for Impala service, you need to uncheck both of the above options. Once done make sure to restart Impala service. CM would show stale and restart icon as well.