Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Setting Hive Auxiliary jars path to HDFS location?

Solved Go to solution
Highlighted

Setting Hive Auxiliary jars path to HDFS location?

Explorer

I want all the jars files like csv jar and json jar should be in hdfs location by that it will load the jar files easily when hive cli is loaded?

My question here is, Can we use HDFS path for setting environment variable like below:

export HIVE_AUX_JARS_PATH=hdfs:///hdp/apps/2.3.4.0-3485/hive/auxjars/*

Does it will load the jar files when we used the HIVE_AUX_JARS_PATH?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Setting Hive Auxiliary jars path to HDFS location?

Rising Star

@Dinesh E You can set this in hive config - so that every Hive/beeline session takes this path

hive.aux.jars.path=hdfs:///hdp/apps/2.3.4.0-3485/hive/auxjars/*

Yes you can use HDFS path here.(recommended to to Use HDFS path... if you go for local path it should also be present in the node where HS2 is running)

https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration

Alternate to it you can also set this:

added this to the script hive-env.sh

  1. HIVE_AUX_JARS_PATH=<path of jar location>

View solution in original post

3 REPLIES 3
Highlighted

Re: Setting Hive Auxiliary jars path to HDFS location?

Rising Star

@Dinesh E You can set this in hive config - so that every Hive/beeline session takes this path

hive.aux.jars.path=hdfs:///hdp/apps/2.3.4.0-3485/hive/auxjars/*

Yes you can use HDFS path here.(recommended to to Use HDFS path... if you go for local path it should also be present in the node where HS2 is running)

https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration

Alternate to it you can also set this:

added this to the script hive-env.sh

  1. HIVE_AUX_JARS_PATH=<path of jar location>

View solution in original post

Highlighted

Re: Setting Hive Auxiliary jars path to HDFS location?

Explorer

Thanks for you reply @Dileep Kumar Chiguruvada.

But my doubt here is that can we use HDFS location for setting the environment variable, bcse in general we will use only linux file system path to set environment variables.

If we use HDFS location, then how the OS will understand that it is a linux file system or HDFS file system?

Re: Setting Hive Auxiliary jars path to HDFS location?

Rising Star

@Dinesh E This depends on the file system we specify in the path(if you dont specify any file system it will take local file system path)

generic: hive.aux.jars.path=file:///hdp/apps/2.3.4.0-3485/hive/auxjars/*

HDFS specific: hive.aux.jars.path=hdfs:///hdp/apps/2.3.4.0-3485/hive/auxjars/*

hope this clears your doubt, if so accept the answer

Don't have an account?
Coming from Hortonworks? Activate your account here