Troubleshoot critical impala query failures more accurately and near real-time using a combination of Cloudera Manager (CM) API, Nifi, and BI tools to:
Recommendations provided later will focus only on the following 10 high-impacting critical query failures:
Query Failure Type
Query failed with a status of Out Of Memory Error.
Query failed with a status of statmissing = true. Indicates missing computing info about table/column level statistics.
Query failed with a status of statscorrupt = true. Indicates corrupt table level block statistics.
Query failed with a status of disk IO error. Indicates extensive disk input/output operations. Could also be related to service/platform level issues.
Query failed with a status of unreachable impala demon. Indicates connectivity and unreachability to impala demons.
Query failed with rpc error. Indicates impala demon connectivity issues.
Queries failed with Canceled exception highlights. Due to query abortion from the user or service end.
Query failed with a status of client inactivity or no active connection.
The query failed with a status of UDF error. Indicates the user's incorrect function or data parsing logic.
rejected from pool
Queries failed with a status of Rejected query form pool.
Note: This approach works for CDP Base and Data Hub, and is not supported for CDW Public or Private cloud.
The implementation steps in the PDF attached to this article will guide you to implement the impala query monitoring and alerting mechanism and build the dashboards.
Critical Impala query exception, Monitor critical Impala query failures, Manage critical Impala query failures, Visibility and Transparency, Gain Actionable insights into Impala query failures, Root cause Impala failures, Impala Query Failures, Identifying critical Impala query failures, Impala query failures.