Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Setting Hive Auxiliary jars path to HDFS location?

avatar
Explorer

I want all the jars files like csv jar and json jar should be in hdfs location by that it will load the jar files easily when hive cli is loaded?

My question here is, Can we use HDFS path for setting environment variable like below:

export HIVE_AUX_JARS_PATH=hdfs:///hdp/apps/2.3.4.0-3485/hive/auxjars/*

Does it will load the jar files when we used the HIVE_AUX_JARS_PATH?

1 ACCEPTED SOLUTION

avatar
Expert Contributor

@Dinesh E You can set this in hive config - so that every Hive/beeline session takes this path

hive.aux.jars.path=hdfs:///hdp/apps/2.3.4.0-3485/hive/auxjars/*

Yes you can use HDFS path here.(recommended to to Use HDFS path... if you go for local path it should also be present in the node where HS2 is running)

https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration

Alternate to it you can also set this:

added this to the script hive-env.sh

  1. HIVE_AUX_JARS_PATH=<path of jar location>

View solution in original post

3 REPLIES 3

avatar
Expert Contributor

@Dinesh E You can set this in hive config - so that every Hive/beeline session takes this path

hive.aux.jars.path=hdfs:///hdp/apps/2.3.4.0-3485/hive/auxjars/*

Yes you can use HDFS path here.(recommended to to Use HDFS path... if you go for local path it should also be present in the node where HS2 is running)

https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration

Alternate to it you can also set this:

added this to the script hive-env.sh

  1. HIVE_AUX_JARS_PATH=<path of jar location>

avatar
Explorer

Thanks for you reply @Dileep Kumar Chiguruvada.

But my doubt here is that can we use HDFS location for setting the environment variable, bcse in general we will use only linux file system path to set environment variables.

If we use HDFS location, then how the OS will understand that it is a linux file system or HDFS file system?

avatar
Expert Contributor

@Dinesh E This depends on the file system we specify in the path(if you dont specify any file system it will take local file system path)

generic: hive.aux.jars.path=file:///hdp/apps/2.3.4.0-3485/hive/auxjars/*

HDFS specific: hive.aux.jars.path=hdfs:///hdp/apps/2.3.4.0-3485/hive/auxjars/*

hope this clears your doubt, if so accept the answer