Created on 11-19-2019 07:40 AM - edited 11-19-2019 08:00 AM
Hello to all,
I'm trying to execute a command
/opt/cloudera/parcels/CDH/lib/spark/bin/spark-beeline -u 'jdbc:hive2:<myserver>:10000' -f /root/testquery.sql
and get an error
Error: Could not find or load main class org.apache.hive.beeline.BeeLine
spark-beeline is a script (taken from spark repository with some path modifications):
set -o posix
export JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera
# Figure out if SPARK_HOME is set
if [ -z "${SPARK_HOME}" ]; then
source "$(dirname "$0")"/find-spark-home
fi
CLASS="org.apache.hive.beeline.BeeLine"
exec "/opt/cloudera/parcels/CDH/lib/spark/bin/spark-class" $CLASS "$@"
I think that I need to add a classpath to Hive Beeline (maybe
/opt/cloudera/parcels/CDH/lib/hive/lib/hive-beeline.jar). I tryed to add
--driver-class-path /opt/cloudera/parcels/CDH/lib/hive/lib/hive-beeline.jar and
--conf "spark.executor.extraClassPath=/opt/cloudera/parcels/CDH/lib/hive/lib/hive-beeline.jar" options to command without results.
What am I doing wrong?
Thanks!
Created 11-19-2019 09:13 AM
After I copied all symlinks /opt/cloudera/parcels/CDH/lib/hive/lib/ to /opt/cloudera/parcels/CDH/lib/spark/jars/, the problem was resolved.
Created on 11-19-2019 08:17 AM - edited 11-19-2019 08:27 AM
Update:
I created a symbolic link to hive-beeline.jar in /opt/cloudera/parcels/CDH/lib/spark/jars/.
Now I get this error:
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: jline/console/history/History
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: jline.console.history.History
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
I also tried to specify the jars path to /opt/cloudera/parcels/CDH/jars - with the same result.
Created 11-19-2019 09:13 AM
After I copied all symlinks /opt/cloudera/parcels/CDH/lib/hive/lib/ to /opt/cloudera/parcels/CDH/lib/spark/jars/, the problem was resolved.