Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Impala Daemon Audit Pipeline Test failed

Impala Daemon Audit Pipeline Test failed

Explorer

Hi,

 

Impala Daemon health check failed due Audit pipeline test failure. Four out of six nodes failed with same error.

 

[13/Sep/2018 16:14:53 +0000] 13226 Monitor-HostMonitor throttling_logger ERROR (30 skipped) Could not find local file system for /var/run/cloudera-scm-agent/process
[13/Sep/2018 16:15:51 +0000] 13226 Audit-Plugin throttling_logger ERROR (313 skipped) Error occurred when sending entry to audit server

 

I'm able to see user queries and other audit information in /var/log/impalad/audit 

 

Users are experiencing performance issues using Impala. Please advice.

 

Thanks,

Meher

3 REPLIES 3

Re: Impala Daemon Audit Pipeline Test failed

Explorer

Found this error message in Navigator audit server log

 

2018-09-14 10:49:51,987 WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper [NavigatorServer-2655]: SQL Error: 1366, SQLState: HY000
2018-09-14 10:49:51,987 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper [NavigatorServer-2655]: Incorrect string value: '\xF0\x9F\x87\xBA\xF0\x9F...' for column 'OPERATION_TEXT' at row 1
2018-09-14 10:49:52,018 WARN com.cloudera.navigator.NavigatorIPCImpl [NavigatorServer-2655]: Error persisting events for service impala

 

Is this a known defect or bug in Impala where user query is unable to insert into operation_text column of audit database table?

 

 

 

Re: Impala Daemon Audit Pipeline Test failed

Master Collaborator

I'd suggest asking in the Cloudera Manager forum - it looks like an issue with some of the CM/Navigator infrastructure rather than anything in Impala itself. I wouldn't expect something like this to affect performance of Impala queries - impala just writes out the logs, everything beyond that is handled by CM/Navigator

Highlighted

Re: Impala Daemon Audit Pipeline Test failed

Explorer

Thank you Tim. I will post it in Cloudera Navigator group as the below errors showed up in Navigator server logs:

 

mysql database characterset is currently set to utf8. It used to have latin1 before. Changed characterset but still the issue persists.

 

Caused by: java.sql.BatchUpdateException: Incorrect string value: '\xF0\x9F\x87\xBA\xF0\x9F...' for column 'OPERATION_TEXT' at row 1
Caused by: java.sql.BatchUpdateException: Incorrect string value: '\xF0\x9F\x92\x98' ...' for column 'OPERATION_TEXT' at row 1
Caused by: java.sql.BatchUpdateException: Incorrect string value: '\xF0\x9F\x93\x9D, ...' for column 'OPERATION_TEXT' at row 1