question Hive Insert query to table(have partitions) failing with vectorization enabled in Archives of Support Questions (Read Only)

Hive Insert query to table(have partitions) failing with vectorization enabled

reddyr211 — Mon, 25 Jun 2018 23:58:03 GMT

Hive Insert query to table(have partitions) failing with vectorization enabled . It was working fine till last week.However it is running fine with vectorization disabled. Can someone help in identifying the issue.

please find the logs with vectorization enabled and disabled/turned off.

Re: Hive Insert query to table(have partitions) failing with vectorization enabled

vmurakami — Tue, 26 Jun 2018 11:50:40 GMT

Hi @Jalender!
Looking at your logs, it seems that Hive with Vectorization cannot process a date value.
Guess you're hitting this -> https://issues.apache.org/jira/browse/HIVE-9862
A clearly and further explanation below (on Limitations part):
https://cwiki.apache.org/confluence/display/Hive/Vectorized+Query+Execution

Could you confirm if there is any date out of the bounds?
Hope this helps!

Re: Hive Insert query to table(have partitions) failing with vectorization enabled

reddyr211 — Tue, 26 Jun 2018 21:50:17 GMT

@Vinicius Higa Murakami Thanks for your reply.

if we enable vectorization it corrupts timestamp values every time or it's intermittent ? because with vectorization enabled it was working fine, we are started to see this happening recently.

Re: Hive Insert query to table(have partitions) failing with vectorization enabled

vmurakami — Tue, 26 Jun 2018 22:12:10 GMT

Hey @Jalender!
Actually, it should work fine indeed.
Basically, you will face issues only when your datetime value is out of the range specified on that link, 1677-09-20 and 2262-04-11. So if you have any datetime more or less of this range, you'll certainly hit that fail msg.
Hope this helps!

Re: Hive Insert query to table(have partitions) failing with vectorization enabled

vmurakami — Tue, 26 Jun 2018 22:13:22 GMT

You don't really need to turn off the vectorization, just need to clean any date outside of the range specified for vectorization 🙂