query causes Impala to crash

[#1]

Hello,

impalad version 2.3.0-cdh5.5.1 RELEASE (build 73bf5bc5afbb47aa7eab06cfbf6023ba8cb74f3c)

this query causes the whole Impala cluster (all impalad daemons) to crash due to an unhandled exception:

select month(cast((`timestamp`) as timestamp)), count(*) FROM cl_qar c group by 1;

immediate output of the query is:

Socket error 104: Connection reset by peer

contents of the error log: terminate called after throwing an instance of 'boost::exception_detail::clone_impl >' what(): Year is out of valid range: 1400..10000

table description is as follows:

ident | string | Inferred from Parquet file

timestamp | bigint | Inferred from Parquet file

cycle_id | bigint | Inferred from Parquet
file_value | string | Inferred from Parquet file

interesting is that when the aggregation is removed then the query does not cause any trouble

the query is never caught by the HistoryServer or Impala Queries in CDM due to the crash

best regards

jaro

[#2 · Accepted solution]

Hi Jaro,

it looks like you could be hitting
https://issues.cloudera.org/browse/IMPALA-2868 which was fixed in Impala
2.5.0. Can you try to disable codegen before running the query and see if
that makes the problem disappear? You can do so by running this command in
a shell before your query: set disable_codegen=1; After that codegen will
be disabled for all queries in that shell only.

Best wishes, Lars

[#3]

This looks like the canonical JIRA for the problem: https://issues.cloudera.org/browse/IMPALA-2184

If you can't upgrade a major version, the JIRA says the fix is also being backported to Impala 2.3.4 (CDH 5.5.4), so if you upgrade to 2.3.4 when that is released, you will get the fix.

[#4]

Thank you very much, Tim and Ivan.

I have tested on customer system and with

"set disable_codegen=1;"

the query does NOT crash the whole impala cluster anymore.

I will inform the customer and provide  him with a link to this discussion