Reply
Highlighted
New Contributor
Posts: 2
Registered: ‎09-02-2016
Accepted Solution

hive.reloadable.aux.jars.path not working properly

 Hi,

I have been struggling with this issue for the past week and running out of options, I hope somebody could help us.

 

We want to use hive.reloadable.aux.jars.path (introduced with HIVE-7553 in Hive 0.14) to put some SerDes and UDFs into HiveServer2's classpath (so users don't need to ADD JAR for every query, and we don't need to restart HS2 to roll out new versions of our libraries).

 

I set  hive.reloadable.aux.jars.path to point at a local directory on HS2's filesystem and it shows up set correctly on the web ui. (hs2:10002/conf)

When connecting to HS2 via beeline, I can now do a show create table on tables using custom serdes, and doing a select * from tbl limit 10 (which basically gives a preview), so our jars have been successfully added to the classpath of our hive session. so far so good!

 

Unfortunately, this doesn't work whenever we run queries that launch mapreduce stages. The query compiles fine and starts, but the Mappers don't have the additional jars added to their classpath so they fail when trying to read from the table.

 

I had a look at  

org.apache.hadoop.hive.ql.exec.mr.ExecDriver

where all the magic is supposed to happen and I cannot figure out how reloadable jars are supposed to be added to the mapred tasks. Seems like it never happens.

 

We are using CDH 5.8.0

Appreciating any help on this! :)

Cloudera Employee
Posts: 177
Registered: ‎03-23-2015

Re: hive.reloadable.aux.jars.path not working properly

Hi,

I believe that you are hitting https://issues.apache.org/jira/browse/HIVE-14037.

HIVE-14037 has been fixed from CDH5.9.0 onwards.

Did you create function with "USING JAR" keywords? If not, please give it a try to see if it will help.

Otherwise, upgrade to latest CDH should help.
Announcements