Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

how to deploy Customized Hive Avro serdes library

avatar
New Member

I have written one Avro ser/des library for Hive to read/write Avro binary data file, which does not contain the Avro schema. And I copied this library into $HIVE/lib.

When I query the Hive external table, which uses my customized ser/des library, (e.g., select * from user_avro), it works as expected.

However if I insert data into that Hive external table like "INSERT OVERWRITE TABLE user_avro SELECT * FROM user), it complains that the my customized ser/des class cannot be found.

This output difference from Hive console is that the second one is launching MapReduce job.

Does anyone know where I need to copy this customized ser/des library except $HIVE/lib?

Many thanks in advance.

1 ACCEPTED SOLUTION

avatar
Super Guru

@yong yang apart from adding jars at each session level as suggested by @Jitendra Yadav you can add them permanently using hive.aux.jars.path

<property>
  <name>hive.aux.jars.path</name>
  <value>/var/lib/hive</value>
</property>

View solution in original post

3 REPLIES 3

avatar
Super Guru

@yong yang

Try using below command on hive shell before executing insert command.

hive>  add jar '\your jar file path';

avatar
Super Guru

@yong yang apart from adding jars at each session level as suggested by @Jitendra Yadav you can add them permanently using hive.aux.jars.path

<property>
  <name>hive.aux.jars.path</name>
  <value>/var/lib/hive</value>
</property>

avatar
New Member

Thanks you very much for your reply. Based on your suggestions, I have added the following into hive-env.sh through Ambari Hive configuration interface:

export HIVE_AUX_JARS_PATH=<my jar full path>

and then restart the hive.

Now it works as expected in both Hive console and Hue BeesWax UI.

Many thanks again.