Trying to get serde working with zeppelin and/or the spark shell. Running the following in either
val links = sqlContext.sql("SELECT * FROM test_links LIMIT 10")
Produces an error
ERROR hive.log: error in initSerDe: java.lang.ClassNotFoundException Class org.apache.hive.hcatalog.data.JsonSerDe not found java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found
Tried added this to /etc/spark/conf/hive-site.xml
But am still getting the error.
This has been solved previously: 1. HIVE_AUX_JARS_PATH in the hive-env.sh or set on your shell (not hive cli) will override the one set by "hive.aux.jars.path" in hive-site.xml. However calling "hive -hiveconf hive.aux.jars.path=..." will override HIVE_AUX_JARS_PATH setting in hive-env.sh. Using "hive --auxpath ..." will append file to any settings you may have in HIVE_AUX_JARS_PATH.2. Since we ship the Apache Hive binaries so hive.reloadable.aux.jars.path should be supported on any hive shipped since HDP 18.104.22.168. "hdfs:///" is supported on add jar, doubt if hive.aux.jars.path supports it.4. Wildcards are not supported in both hive.aux.jars.path and add jar. See previous post by Deepesh (looks like a HWX worker) at https://community.hortonworks.com/questions/2390/methods-to-add-jars-to-hive.html