Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

hive.reloadable.aux.jars.path not working properly

avatar
New Contributor

 Hi,

I have been struggling with this issue for the past week and running out of options, I hope somebody could help us.

 

We want to use hive.reloadable.aux.jars.path (introduced with HIVE-7553 in Hive 0.14) to put some SerDes and UDFs into HiveServer2's classpath (so users don't need to ADD JAR for every query, and we don't need to restart HS2 to roll out new versions of our libraries).

 

I set  hive.reloadable.aux.jars.path to point at a local directory on HS2's filesystem and it shows up set correctly on the web ui. (hs2:10002/conf)

When connecting to HS2 via beeline, I can now do a show create table on tables using custom serdes, and doing a select * from tbl limit 10 (which basically gives a preview), so our jars have been successfully added to the classpath of our hive session. so far so good!

 

Unfortunately, this doesn't work whenever we run queries that launch mapreduce stages. The query compiles fine and starts, but the Mappers don't have the additional jars added to their classpath so they fail when trying to read from the table.

 

I had a look at  

org.apache.hadoop.hive.ql.exec.mr.ExecDriver

where all the magic is supposed to happen and I cannot figure out how reloadable jars are supposed to be added to the mapred tasks. Seems like it never happens.

 

We are using CDH 5.8.0

Appreciating any help on this! 🙂

1 ACCEPTED SOLUTION

avatar
Super Guru
Hi,

I believe that you are hitting https://issues.apache.org/jira/browse/HIVE-14037.

HIVE-14037 has been fixed from CDH5.9.0 onwards.

Did you create function with "USING JAR" keywords? If not, please give it a try to see if it will help.

Otherwise, upgrade to latest CDH should help.

View solution in original post

1 REPLY 1

avatar
Super Guru
Hi,

I believe that you are hitting https://issues.apache.org/jira/browse/HIVE-14037.

HIVE-14037 has been fixed from CDH5.9.0 onwards.

Did you create function with "USING JAR" keywords? If not, please give it a try to see if it will help.

Otherwise, upgrade to latest CDH should help.