hive.reloadable.aux.jars.path not working properly

zaboron — Fri, 16 Sep 2022 12:05:16 GMT

Hi,

I have been struggling with this issue for the past week and running out of options, I hope somebody could help us.

We want to use hive.reloadable.aux.jars.path (introduced with HIVE-7553 in Hive 0.14) to put some SerDes and UDFs into HiveServer2's classpath (so users don't need to ADD JAR for every query, and we don't need to restart HS2 to roll out new versions of our libraries).

I set hive.reloadable.aux.jars.path to point at a local directory on HS2's filesystem and it shows up set correctly on the web ui. (hs2:10002/conf)

When connecting to HS2 via beeline, I can now do a show create table on tables using custom serdes, and doing a select * from tbl limit 10 (which basically gives a preview), so our jars have been successfully added to the classpath of our hive session. so far so good!

Unfortunately, this doesn't work whenever we run queries that launch mapreduce stages. The query compiles fine and starts, but the Mappers don't have the additional jars added to their classpath so they fail when trying to read from the table.

I had a look at

org.apache.hadoop.hive.ql.exec.mr.ExecDriver

where all the magic is supposed to happen and I cannot figure out how reloadable jars are supposed to be added to the mapred tasks. Seems like it never happens.

We are using CDH 5.8.0

Appreciating any help on this! 🙂

Re: hive.reloadable.aux.jars.path not working properly

EricL — Mon, 21 Aug 2017 05:33:49 GMT

Hi,

I believe that you are hitting https://issues.apache.org/jira/browse/HIVE-14037.

HIVE-14037 has been fixed from CDH5.9.0 onwards.

Did you create function with "USING JAR" keywords? If not, please give it a try to see if it will help.

Otherwise, upgrade to latest CDH should help.

question Re: hive.reloadable.aux.jars.path not working properly in Archives of Support Questions (Read Only)

hive.reloadable.aux.jars.path not working properly

Re: hive.reloadable.aux.jars.path not working properly