Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

spark-shell command not finding correct Java JDK path.

spark-shell command not finding correct Java JDK path.

Expert Contributor

I am trying to run the spark-shell with the following command. Spark shell is under /usr/hdp/current/spark-client/bin

# spark-shell

but I am getting the following error:

/usr/hdp/2.4.0.0-169/spark/bin/spark-class: line 101: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.77-0.b03.el6_7.x86_64/jre/bin/java: No such file or directory

I have checked the JDK path and found that the JDK is installed under /usr/jdk/jdk1.8.0_121 directory

my JAVA_HOME is set to /usr/jdk/jdk1.8.0_121

I would like to undestand why spark is not getting the right path to JAVA_HOME

1 REPLY 1

Re: spark-shell command not finding correct Java JDK path.

Guru

@Pradeep kumar, you should have JAVA_HOME set in spark-env.sh. Can you please check what is the value of JAVA_HOME in spark-env.sh?

Don't have an account?
Coming from Hortonworks? Activate your account here