question Re: (hive) How to permanently add SerDe JARs to Hive? in Support Questions

(hive) How to permanently add SerDe JARs to Hive?

sroberts — Thu, 29 Oct 2015 23:59:19 GMT

Regarding permanently adding custom JARs for Hive SerDes

(Feel free to answer those which you now, and come back later to answer the rest.)

What is the difference between these, and do they conflict?
1. hive-env -> hive-env template -> HIVE_AUX_JARS_PATH
2. hive-site -> hive.aux.jars.path
Is hive.reloadable.aux.jars.path supported?
Is hdfs:// supported as the path?
Are wildcards or open paths supported?
1. e.g. /mydir/*.jar or /mydir
What other permanent methods are supported?
What other temporary methods are supported?
1. Other than `add jar jarfile.jar`

Re: (hive) How to permanently add SerDe JARs to Hive?

deepesh1 — Fri, 30 Oct 2015 01:33:01 GMT

Here are some of the answers:

1. HIVE_AUX_JARS_PATH in the hive-env.sh or set on your shell (not hive cli) will override the one set by "hive.aux.jars.path" in hive-site.xml. However calling "hive -hiveconf hive.aux.jars.path=..." will override HIVE_AUX_JARS_PATH setting in hive-env.sh. Using "hive --auxpath ..." will append file to any settings you may have in HIVE_AUX_JARS_PATH.

2. Since we ship the Apache Hive binaries so hive.reloadable.aux.jars.path should be supported on any hive shipped since HDP 2.2.0.

3. "hdfs:///" is supported on add jar, doubt if hive.aux.jars.path supports it.

4. Wildcards are not supported in both hive.aux.jars.path and add jar.

Re: (hive) How to permanently add SerDe JARs to Hive?

nsabharwal — Fri, 30 Oct 2015 02:54:52 GMT

@Sean Roberts

#5 I hope this helps

HS2 should support .hiverc

Global Init File

A global init file can be placed in the configured hive.server2.global.init.file.location location (Hive 0.14 onward, see HIVE-5160, HIVE-7497, and HIVE-8138). This can be either the path to the init file itself, or a directory where an init file named ".hiverc" is expected.

The init file lists a set of commands that will run for users of this HiveServer2 instance, such as register a standard set of jars and functions.

Re: (hive) How to permanently add SerDe JARs to Hive?

gbraccialli3 — Fri, 30 Oct 2015 11:18:20 GMT

I put jars into folder /usr/hdp/current/hive-client/auxlib (in each hiveserver2/hive cli node).

Other solutions might work, but in my experience was when it worked for hiveserver2, it did not work for hive cli, or good for hue and bad for squirel, so I gave up and started using auxlib.