Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

query causes Impala to crash

Explorer

Hello,

 

impalad version 2.3.0-cdh5.5.1 RELEASE (build 73bf5bc5afbb47aa7eab06cfbf6023ba8cb74f3c)

 

this query causes the whole Impala cluster (all impalad daemons) to crash due to an unhandled exception:

 

select month(cast((`timestamp`) as timestamp)), count(*) FROM cl_qar c group by 1;

 

immediate output of the query is:

Socket error 104: Connection reset by peer

 

contents of the error log: terminate called after throwing an instance of 'boost::exception_detail::clone_impl >' what(): Year is out of valid range: 1400..10000

 

table description is as follows:

ident | string | Inferred from Parquet file

timestamp | bigint | Inferred from Parquet file

cycle_id | bigint | Inferred from Parquet
file_value | string | Inferred from Parquet file

 

interesting is that when the aggregation is removed then the query does not cause any trouble

 

the query is never caught by the HistoryServer or Impala Queries in CDM due to the crash

 

best regards

jaro

1 ACCEPTED SOLUTION

Expert Contributor
Hi Jaro,

it looks like you could be hitting
https://issues.cloudera.org/browse/IMPALA-2868 which was fixed in Impala
2.5.0. Can you try to disable codegen before running the query and see if
that makes the problem disappear? You can do so by running this command in
a shell before your query: set disable_codegen=1; After that codegen will
be disabled for all queries in that shell only.

Best wishes, Lars

View solution in original post

3 REPLIES 3

Expert Contributor
Hi Jaro,

it looks like you could be hitting
https://issues.cloudera.org/browse/IMPALA-2868 which was fixed in Impala
2.5.0. Can you try to disable codegen before running the query and see if
that makes the problem disappear? You can do so by running this command in
a shell before your query: set disable_codegen=1; After that codegen will
be disabled for all queries in that shell only.

Best wishes, Lars

This looks like the canonical JIRA for the problem: https://issues.cloudera.org/browse/IMPALA-2184


If you can't upgrade a major version, the JIRA says the fix is also being backported to Impala 2.3.4 (CDH 5.5.4), so if you upgrade to 2.3.4 when that is released, you will get the fix.

Explorer

Thank you very much, Tim and Ivan.

 

I have tested on customer system and with

"set disable_codegen=1;"

the query does NOT crash the whole impala cluster anymore.

 

I will inform the customer and provide  him with a link to this discussion