Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark LLAP Custom Hive Serde Class Not found Error

Spark LLAP Custom Hive Serde Class Not found Error

New Contributor

We have created a custom hive serde which is added to hive aux lib in all hive server nodes. It was working fine in hive cli queries. When Queried through Spark connected hive through LLAP , we are getting class not found error for the custom serde class if we try to insert the data to another table.

1) For the Custome Serde , Create table and Select Query is working fine in Spark LLAP. But in some cases data selected is only 1000 records even though it has many records in the table create using serde but in hive cli it returns all records.

2) If we try to insert the data in serde table to another table it throw class not found error. (Query : hive.executeUpdate("Insert into orctable select * from serdetable"). It works fine in hive cli.

	Caused by: java.lang.ClassNotFoundException: **.CustomSerde

	 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154)

Please help to resolve this issue.

Don't have an account?
Coming from Hortonworks? Activate your account here