09-14-2018 06:36 AM
Impala Daemon health check failed due Audit pipeline test failure. Four out of six nodes failed with same error.
[13/Sep/2018 16:14:53 +0000] 13226 Monitor-HostMonitor throttling_logger ERROR (30 skipped) Could not find local file system for /var/run/cloudera-scm-agent/process
[13/Sep/2018 16:15:51 +0000] 13226 Audit-Plugin throttling_logger ERROR (313 skipped) Error occurred when sending entry to audit server
I'm able to see user queries and other audit information in /var/log/impalad/audit
Users are experiencing performance issues using Impala. Please advice.
09-14-2018 07:53 AM
Found this error message in Navigator audit server log
2018-09-14 10:49:51,987 WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper [NavigatorServer-2655]: SQL Error: 1366, SQLState: HY000
2018-09-14 10:49:51,987 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper [NavigatorServer-2655]: Incorrect string value: '\xF0\x9F\x87\xBA\xF0\x9F...' for column 'OPERATION_TEXT' at row 1
2018-09-14 10:49:52,018 WARN com.cloudera.navigator.NavigatorIPCImpl [NavigatorServer-2655]: Error persisting events for service impala
Is this a known defect or bug in Impala where user query is unable to insert into operation_text column of audit database table?
09-14-2018 09:51 AM
I'd suggest asking in the Cloudera Manager forum - it looks like an issue with some of the CM/Navigator infrastructure rather than anything in Impala itself. I wouldn't expect something like this to affect performance of Impala queries - impala just writes out the logs, everything beyond that is handled by CM/Navigator
09-17-2018 07:03 PM
Thank you Tim. I will post it in Cloudera Navigator group as the below errors showed up in Navigator server logs:
mysql database characterset is currently set to utf8. It used to have latin1 before. Changed characterset but still the issue persists.
Caused by: java.sql.BatchUpdateException: Incorrect string value: '\xF0\x9F\x87\xBA\xF0\x9F...' for column 'OPERATION_TEXT' at row 1
Caused by: java.sql.BatchUpdateException: Incorrect string value: '\xF0\x9F\x92\x98' ...' for column 'OPERATION_TEXT' at row 1
Caused by: java.sql.BatchUpdateException: Incorrect string value: '\xF0\x9F\x93\x9D, ...' for column 'OPERATION_TEXT' at row 1