Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

[CDH 5.3] Hive UDF Sentry problem

[CDH 5.3] Hive UDF Sentry problem


I have created a simple java class as UDF


@UDFType(deterministic = true)
value="returns hello + input",
extended="Example usage: SELECT bla('sap');"
class MyHelloUdf extends UDF {

public Text evaluate(Text input) {
if(input == null) return null;
return new Text("hello" + input.toString());

I build a jar using mvn assembly:single. Which created the test-1-jar-with-dependencies.jar


We have a CDH 5.3 cluster with Sentry and Kerberos. So I followed the Cloudera procedure:


On hiveserver2 create dir on local filesystem   /tmp/udfs/   and put jar here (access rights to 777)

Added this dir in  Hive Auxiliary JARs Directory, through CM.

Put the jar in my home folder on HDFS, rights 777 again.

Updated the sentry-provider.ini although I am an 'admin'.


CREATE FUNCTION bla AS 'MyHelloUdf' USING JAR 'hdfs:///user/alexanderbij/test-1-jar-with-dependencies.jar';


On HiveServer2 log i see:

2015-02-26 13:43:42,077 INFO org.apache.hadoop.hive.ql.Driver: Starting command: CREATE FUNCTION bla AS 'MyHelloUdf' USING JAR 'hdfs:///user/alexanderbij/test-udfs-1-jar-with-
2015-02-26 13:50:36,716 INFO SessionState: converting to local hdfs:///user/alexanderbij/test-udfs-1-jar-with-dependencies.jar
2015-02-26 13:50:36,732 INFO SessionState: Added /tmp/5d79e1db-7099-453b-82c4-351d07bd4d49_resources/test-udfs-1-jar-with-dependencies.jar to class path
2015-02-26 13:50:36,732 INFO SessionState: Added resource: /tmp/5d79e1db-7099-453b-82c4-351d07bd4d49_resources/test-udfs-1-jar-with-dependencies.jar


When I query:

describe function extended bla;

I get results from the annotation @Description, that works, the jar seems to be available.


But when I want to use the function:


select bla('helpme');


HiveServer2 log:

015-02-26 14:20:47,827 INFO hive.ql.Context: New scratch dir is hdfs://
2015-02-26 14:20:47,830 INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer: Completed getting MetaData in Semantic Analysis
2015-02-26 14:20:47,831 INFO hive.ql.Context: New scratch dir is hdfs://
2015-02-26 14:20:48,100 ERROR org.apache.hadoop.hive.ql.Driver: FAILED: SemanticException [Error 10014]: Line 1:7 Wrong arguments ''helpme'': The UDF implementation class 'MyHelloUdf' is no
t present in the class path
org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:7 Wrong arguments ''helpme'': The UDF implementation class 'MyHelloUdf' is not present in the class path
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(
at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
at org.apache.hadoop.hive.ql.Driver.compile(
at org.apache.hadoop.hive.ql.Driver.compile(
at org.apache.hadoop.hive.ql.Driver.compileInternal(
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(
at org.apache.hive.service.cli.operation.SQLOperation.prepare(
at org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(
at org.apache.hive.service.cli.CLIService.executeStatementAsync(
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(


I did not copy the jar to any worker nodes, only on HiveServer2 and on HDFS.


What did I miss?


Re: [CDH 5.3] Hive UDF Sentry problem

Cloudera Employee



Does it work without Sentry enabled ?





Re: [CDH 5.3] Hive UDF Sentry problem


I'm not in the position to try that at the moment, but I'll ask my colleags.


Re: [CDH 5.3] Hive UDF Sentry problem


Hi ,


After copying jar to hs2 local directories (Aux dir path) and granting 777 to local and hdfs URI from sentry ,UDF creation didnt work .Full hive service restart was ateempted and after that UDF worked.

Re: [CDH 5.3] Hive UDF Sentry problem

Thanks, although I posted this more than 1y ago. I assume its working properly now.

The docs explain the process very clairly:

At the bottom, a restart from HS2 is mandatory.