question Re: how to add jar files through hive view on ambari in Support Questions

how to add jar files through hive view on ambari

mscferenzo — Thu, 28 Jul 2016 16:57:25 GMT

hello all

I am very new to hadoop and I wanted to add a couple of jar files to be able to perform my queries

I tried using add jars and the path of the jars like I do in the terminal but I get an error

add jar /tmp/udfs/esri-geometry-api.jar /tmp/udfs/spatial-sdk-hadoop.jar;

Error while compiling statement: FAILED: ParseException line 4:0 cannot recognize input near 'add' 'jar' '/' [ERROR_STATUS]

since I am a newbie i dont exactly know how to add them since the path where the jars are located is right, and I am able to do that successfully on the terminal throught he back end.

Kindly if someone could elaborately explain me how to add these jar files, i would be eternally grateful

Re: how to add jar files through hive view on ambari

jyadav — Thu, 28 Jul 2016 17:01:38 GMT

Hi, can you please try to add those jars saparetly?

add jar /tmp/udfs/esri-geometry-api.jar

add jar /tmp/udfs/spatial-sdk-hadoop.jar

Re: how to add jar files through hive view on ambari

mscferenzo — Thu, 28 Jul 2016 17:04:40 GMT

Hi, Thanks

I tried to do so, still showing the same error..

Re: how to add jar files through hive view on ambari

sbhat — Thu, 28 Jul 2016 17:05:43 GMT

Can you check if this helps? https://community.hortonworks.com/questions/18727/how-do-i-add-filesjars-through-hive-view.html

Re: how to add jar files through hive view on ambari

jyadav — Thu, 28 Jul 2016 17:11:24 GMT

ok, then you can also put these required jars inside hive.aux.jars.path through ambari hive conf, jars will get picked automatically.

Re: how to add jar files through hive view on ambari

mscferenzo — Thu, 28 Jul 2016 17:12:05 GMT

Hi,

before posting this question, i did refer the above link, and so managed to add the jar files through the files view into /tmp/udf. But I dont want to create a temporary function, I just want to add the couple of jars because they are needed for any query to run because it is geospatial data.

thanks

Re: how to add jar files through hive view on ambari

mscferenzo — Thu, 28 Jul 2016 17:59:55 GMT

Hi jitendra,

So i set this property on custom hive-server2 as hive.aux.jars.path and the path right??

and do i have to set 2 different properties for both the jars or use a ',' to seperate both the jars in a single property?

Re: how to add jar files through hive view on ambari

jyadav — Thu, 28 Jul 2016 18:11:03 GMT

mkdir /usr/hdp/current/hive-server2/auxlib/ on HS2 node.

copy required jars to /usr/hdp/current/hive-server2/auxlib/ loaction.

Added below property into custom hive-server2 and restart the all affected services. And see if that help.

<property>
  <name>hive.aux.jars.path</name>
  <value>/usr/hdp/current/hive-server2/auxlib/</value>
</property>

Re: how to add jar files through hive view on ambari

mscferenzo — Thu, 28 Jul 2016 18:39:44 GMT

hi Jitendra,

Tried to do as you said

when I tried to execute the query say select * from test.building i got the error

Error while compiling statement: FAILED: RuntimeException java.lang.ClassNotFoundException: com.esri.json.hadoop.UnenclosedJsonInputFormat [ERROR_STATUS]

Re: how to add jar files through hive view on ambari

jyadav — Thu, 28 Jul 2016 18:59:56 GMT

Looks like it still didn't picked the jar file, lets try another approach.

In hive-env.sh file through Ambari.

export HIVE_AUX_JARS_PATH=<jar1 file path>,<jar2 file path>

And restart the services.

Re: how to add jar files through hive view on ambari

mscferenzo — Thu, 28 Jul 2016 20:05:15 GMT

Did you mean hive-env template in advance hive-env?

Re: how to add jar files through hive view on ambari

jyadav — Thu, 28 Jul 2016 20:11:31 GMT

Yes, under same location. just after below line

"# Folder containing extra libraries required for hive compilation/execution can be controlled by:

Re: how to add jar files through hive view on ambari

mscferenzo — Thu, 28 Jul 2016 20:23:17 GMT

no luck. here is what is visible initially and i added the path below those lines. Restarted it, but still getting the same error

Folder containing extra libraries required for hive compilation/execution can be controlled by: if [ "${HIVE_AUX_JARS_PATH}" != "" ]; then if [ -f "${HIVE_AUX_JARS_PATH}" ]; then export HIVE_AUX_JARS_PATH=${HIVE_AUX_JARS_PATH} elif [ -d "/usr/hdp/current/hive-webhcat/share/hcatalog" ]; then export HIVE_AUX_JARS_PATH=/usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar fi elif [ -d "/usr/hdp/current/hive-webhcat/share/hcatalog" ]; then export HIVE_AUX_JARS_PATH=/usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar fi export METASTORE_PORT={{hive_metastore_port}}

export HIVE_AUX_JARS_PATH=/usr/hdp/current/hive-server2/auxlib/esri-geometry-api.jar,/usr/hdp/current/hive-server2/auxlib/spatial-sdk-hadoop.jar.

I am not sure whether I am making any mistake

Should I try changing it in hive-site.xml or hive-env.sh through the terminal/shell, will it make any difference?

I was referring to this blog

http://chetnachaudhari.github.io/2016-02-16/how-to-add-auxiliary-jars-in-hive/

Re: how to add jar files through hive view on ambari

jyadav — Thu, 28 Jul 2016 20:27:28 GMT

Manual change won't work becuase when you restart the service through Ambari, confs it will get overwrite. I believe only change remaining is adding in hive-site.xml through Ambari and restart service.

Re: how to add jar files through hive view on ambari

mscferenzo — Thu, 28 Jul 2016 20:40:44 GMT

i dont think i see where i can add or edit the property through ambari in advanced hive-site or custom hive-site

Re: how to add jar files through hive view on ambari

cstanca — Fri, 29 Jul 2016 00:42:23 GMT

@Srinivas Santhanam

Add semicolon at the end and make sure that your ambari user is mapped to an OS user that has access to the path

add jar /tmp/udfs/esri-geometry-api.jar;

add jar /tmp/udfs/spatial-sdk-hadoop.jar;

My suggestion is to place these libraries in HDFS with your Ambari user that has hdfs priviles. Such you can have access to the libraries from any node with hive client.

Example:

add jar hdfs://whateverhostname:8020/tmp/esri/esri-geometry-api.jar;

If the response addresses your problem, don't forget to vote and accept best answer. It is a stimulus for the effort. Thank you.