Skip to main content
Snussi
Snussi
Gold ⭐️
November 19, 2019
Solved

spark-class Could not find or load main class org.apache.hive.beeline.BeeLine

Hello to all,

 

I'm trying to execute a command

 

/opt/cloudera/parcels/CDH/lib/spark/bin/spark-beeline -u 'jdbc:hive2:<myserver>:10000' -f /root/testquery.sql

 

and get an error

 

Error: Could not find or load main class org.apache.hive.beeline.BeeLine

 

spark-beeline is a script (taken from spark repository with some path modifications):

 

set -o posix
export JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera
# Figure out if SPARK_HOME is set
if [ -z "${SPARK_HOME}" ]; then
  source "$(dirname "$0")"/find-spark-home
fi
CLASS="org.apache.hive.beeline.BeeLine"
exec "/opt/cloudera/parcels/CDH/lib/spark/bin/spark-class" $CLASS "$@"

 

 

I think that I need to add a classpath to Hive Beeline (maybe

/opt/cloudera/parcels/CDH/lib/hive/lib/hive-beeline.jar). I tryed to add 

--driver-class-path /opt/cloudera/parcels/CDH/lib/hive/lib/hive-beeline.jar and 

--conf "spark.executor.extraClassPath=/opt/cloudera/parcels/CDH/lib/hive/lib/hive-beeline.jar" options to command without results.

 

What am I doing wrong?

Thanks!

Snussi
SnussiAnswer
Gold ⭐️
November 19, 2019

After I copied all symlinks /opt/cloudera/parcels/CDH/lib/hive/lib/ to /opt/cloudera/parcels/CDH/lib/spark/jars/, the problem was resolved.

View in thread

2 replies

Snussi
SnussiAuthor
Gold ⭐️
November 19, 2019

Update:

 

I created a symbolic link to hive-beeline.jar in /opt/cloudera/parcels/CDH/lib/spark/jars/.

Now I get this error:

 

Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: jline/console/history/History
	at java.lang.Class.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
	at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
	at java.lang.Class.getMethod0(Class.java:3018)
	at java.lang.Class.getMethod(Class.java:1784)
	at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: jline.console.history.History
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

I also tried to specify the jars path to /opt/cloudera/parcels/CDH/jars - with the same result.

 

Translate
Snussi
SnussiAuthorAnswer
Gold ⭐️
November 19, 2019

After I copied all symlinks /opt/cloudera/parcels/CDH/lib/hive/lib/ to /opt/cloudera/parcels/CDH/lib/spark/jars/, the problem was resolved.

Translate
Terms & ConditionsAccessibility statement