Skip to main content
rajkumar_singh
Gold ⭐️
rajkumar_singh
Gold ⭐️
August 2, 2017

Creating custom udf and adding udf jar to Hive LLAP

  • October 5, 2026 at 9:19 AM
  • 0 replies
  • 4415 views
  • Translate All

Creating and running Temporary functions are discouraged while running a query on LLAP because of security reason since many users are sharing same instances of LLAP, it can create a conflict but still, you can create temp functions using add jar and hive.llap.execution.mode=auto.

with exclusive llap execution mode(hive.llap.execution.mode=only) you will run into the ClassNotFoundException, hive.llap.execution.mode=auto will allow some part of query(map tasks) to run in the tez container.

Here are steps to create a custom permanent function in LLAP(steps are tested on HDP-260)

1. create a jar for UDF function (in this case I am using simple udf):

git clone https://github.com/rajkrrsingh/SampleCode

mvn clean package

2. upload the target/SampleCode.jar to the node where HSI is running(in my case I have copied it to /tmp directory)

3. add jar to hive_aux_jars (goto Ambari--> hive --> config --> hive-interactive-env template)

export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH:/tmp/SampleCode.jar

4. add the jar to Auxillary JAR list (goto Ambari--> hive --> config --> Auxillary JAR list)

Auxillary JAR list=/tmp/SampleCode.jar

5. restart LLAP

6. create Permanent Custom function

connect to HSI using beeline

create FUNCTION CustomLength as 'com.rajkrrsingh.hiveudf.CustomLength';

describe function CustomLength;

select CustomLength(description) from sample_07 limit 1;

7. check where the SampleCode.jar localized

root@hdp26 container_e06_1501140901077_0019_01_000002]# pwd
/hadoop/yarn/local/usercache/hive/appcache/application_1501140901077_0019/container_e06_1501140901077_0019_01_000002
[root@hdp26 container_e06_1501140901077_0019_01_000002]# find . -iname sample*
./app/install/lib/SampleCode.jar

    Terms & ConditionsAccessibility statement