question Re: CodeGen way to slow in Archives of Support Questions (Read Only)

CodeGen way to slow

maurin — Fri, 16 Sep 2022 11:29:52 GMT

Hi,
using impala 2.7(8) with cdh5.10.1 here.
I am trying a simple query :
`select distinct(date_col_partition) from table_1`

and it is taking 20 sec.
But When I do a set DISABLE_CODEGEN=true;

It take only less than a second.

here is the profle gist: https://gist.github.com/anonymous/1a5faa3a10d4495f7b8abc3c964457db

Any idea of what is going wrong?

thanks

Re: CodeGen way to slow

alex.behm — Fri, 21 Apr 2017 04:42:20 GMT

Hi Maurin,

thanks for posting, this is pretty interesting. What is the type of your "cuberon_event_date" column?

Alex

Re: CodeGen way to slow

alex.behm — Fri, 21 Apr 2017 04:45:50 GMT

As an experiment, it would be interesting to try the query with the same data using a different data format, e.g., text. You can do a quick CREATE TABLE test as SELECT * FROM <original_table> and the retry the query.

Re: CodeGen way to slow

maurin — Fri, 21 Apr 2017 04:57:55 GMT

It is a string of that look like "YYYY-MM-DD"
the table is stored as avro. I can try using parquet or text if you want

Re: CodeGen way to slow

alex.behm — Fri, 21 Apr 2017 05:06:36 GMT

Thanks. Trying Parquet would help. Just want to see of the high optimization time in codegen is due to some glitch for Avro.

Re: CodeGen way to slow

Tim Armstrong — Fri, 21 Apr 2017 05:27:22 GMT

Does the table have a lot of columns or anything unusual like that?

Re: CodeGen way to slow

maurin — Fri, 21 Apr 2017 06:10:05 GMT

it seems to be coming from avro.
I created the table as parquet and it took 0.48sec.
The table have about 900 columns, so nothing to fancy.

thanks

Re: CodeGen way to slow

alex.behm — Fri, 21 Apr 2017 06:14:02 GMT

Thanks for investigating. We've confirmed internally that the issue is related to Avro with many columns. 900 is somewhat wide.

Thanks for reporting! We'll continue to look into this issue.

Re: CodeGen way to slow

maurin — Fri, 21 Apr 2017 08:32:31 GMT

thanks!
If you open a jira, can you send me the link?
I will probably disable codegen for now. And wait until you push a fix to re enable it.
thanks

Re: CodeGen way to slow

alex.behm — Fri, 21 Apr 2017 14:50:50 GMT

https://issues.apache.org/jira/browse/IMPALA-5243