Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

class not found exception while running UDF in impala

avatar
Explorer

Hi,

I am running UDF which has sql query .UDF is running fine in Hive.While running in Impala I am getting the below exception.

 


Prettytable cannot resolve string columns values that have  embedded tabs. Reverting to tab delimited text output
++++java.lang.ClassNotFoundException: org.apache.hive.jdbc.HiveDriver
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:190)
    at GetData.evaluate(GetData.java:53)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.impala.hive.executor.UdfExecutor.evaluate(UdfExecutor.java:353)
    at org.apache.impala.hive.executor.UdfExecutor.evaluate(UdfExecutor.java:288)
    at org.apache.impala.service.FeSupport.NativeEvalExprsWithoutRow(Native Method)
    at org.apache.impala.service.FeSupport.EvalExprsWithoutRow(FeSupport.java:172)
    at org.apache.impala.service.FeSupport.EvalExprWithoutRow(FeSupport.java:130)
    at org.apache.impala.analysis.LiteralExpr.create(LiteralExpr.java:178)
    at org.apache.impala.rewrite.FoldConstantsRule.apply(FoldConstantsRule.java:68)
    at org.apache.impala.rewrite.ExprRewriter.applyRuleBottomUp(ExprRewriter.java:85)
    at org.apache.impala.rewrite.ExprRewriter.applyRuleRepeatedly(ExprRewriter.java:71)
    at org.apache.impala.rewrite.ExprRewriter.rewrite(ExprRewriter.java:55)
    at org.apache.impala.analysis.SelectList.rewriteExprs(SelectList.java:97)
    at org.apache.impala.analysis.SelectStmt.rewriteExprs(SelectStmt.java:886)
    at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:396)
    at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:368)
    at org.apache.impala.service.Frontend.analyzeStmt(Frontend.java:903)
    at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1052)
    at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:156)

 

I changed driver to  impala driver and tested but still same error with class not found exception on impala driver is coming.

 

Request your inputs or suggestions.

 

Thank You

1 ACCEPTED SOLUTION

avatar

It looks like it's referencing a Java class "org.apache.hive.jdbc.HiveDriver" that's not available on the Impala JVM's classpath. In Hive I assume that that class is deliberately or accidentally on the classpath of the java tasks spawned for the Hive job. Impala doesn't have that class on it's classpath.

 

It seems pretty weird for a UDF to depend on a use a database driver, so my first thought is to modify the UDF to removed the need for the dependency. Otherwise, generally, if you want to use Java classes from your UDF they would need to be built into the UDF JAR.

View solution in original post

2 REPLIES 2

avatar

It looks like it's referencing a Java class "org.apache.hive.jdbc.HiveDriver" that's not available on the Impala JVM's classpath. In Hive I assume that that class is deliberately or accidentally on the classpath of the java tasks spawned for the Hive job. Impala doesn't have that class on it's classpath.

 

It seems pretty weird for a UDF to depend on a use a database driver, so my first thought is to modify the UDF to removed the need for the dependency. Otherwise, generally, if you want to use Java classes from your UDF they would need to be built into the UDF JAR.

avatar
Explorer

Thank You.It worked.