question spark-class Could not find or load main class org.apache.hive.beeline.BeeLine in Support Questions

spark-class Could not find or load main class org.apache.hive.beeline.BeeLine

Snussi — Tue, 19 Nov 2019 16:00:17 GMT

Hello to all,

I'm trying to execute a command

/opt/cloudera/parcels/CDH/lib/spark/bin/spark-beeline -u 'jdbc:hive2:<myserver>:10000' -f /root/testquery.sql

and get an error

Error: Could not find or load main class org.apache.hive.beeline.BeeLine

spark-beeline is a script (taken from spark repository with some path modifications):

set -o posix export JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera # Figure out if SPARK_HOME is set if [ -z "${SPARK_HOME}" ]; then source "$(dirname "$0")"/find-spark-home fi CLASS="org.apache.hive.beeline.BeeLine" exec "/opt/cloudera/parcels/CDH/lib/spark/bin/spark-class" $CLASS "$@"

I think that I need to add a classpath to Hive Beeline (maybe

/opt/cloudera/parcels/CDH/lib/hive/lib/hive-beeline.jar). I tryed to add

--driver-class-path /opt/cloudera/parcels/CDH/lib/hive/lib/hive-beeline.jar and

--conf "spark.executor.extraClassPath=/opt/cloudera/parcels/CDH/lib/hive/lib/hive-beeline.jar" options to command without results.

What am I doing wrong?

Thanks!

Re: spark-class Could not find or load main class org.apache.hive.beeline.BeeLine

Snussi — Tue, 19 Nov 2019 16:27:29 GMT

Update:

I created a symbolic link to hive-beeline.jar in /opt/cloudera/parcels/CDH/lib/spark/jars/.

Now I get this error:

Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.NoClassDefFoundError: jline/console/history/History at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.privateGetMethodRecursive(Class.java:3048) at java.lang.Class.getMethod0(Class.java:3018) at java.lang.Class.getMethod(Class.java:1784) at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) Caused by: java.lang.ClassNotFoundException: jline.console.history.History at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

I also tried to specify the jars path to /opt/cloudera/parcels/CDH/jars - with the same result.

Re: spark-class Could not find or load main class org.apache.hive.beeline.BeeLine

Snussi — Tue, 19 Nov 2019 17:13:10 GMT

After I copied all symlinks /opt/cloudera/parcels/CDH/lib/hive/lib/ to /opt/cloudera/parcels/CDH/lib/spark/jars/, the problem was resolved.