Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Unable to launch spark using spark-shell on linux

SOLVED Go to solution

Unable to launch spark using spark-shell on linux

Contributor

I am trying to set up SPARK2 on my cloudera cluster. For that, I have JDK1.8:

 

 

I have installed scala 2.11.8 using the rpm file:

I have downloaded, extracted the spark version 2.2.0 on my home directory: /home/cloudera.

 

I made changes to the PATH variable in .bashrc as below:

 

But when I try to execute spark-shell from the home directory: /home/cloudera, it says no such file or directory which can be seen below:

 

[cloudera@quickstart ~]$ spark-shell
/home/cloudera/spark/bin/spark-class: line 71: /usr/java/jdk1.7.0_67-cloudera/bin/java: No such file or directory
[cloudera@quickstart ~]$

 

Could anyone let me know how can I fix the problem and configure it properly ?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Unable to launch spark using spark-shell on linux

Expert Contributor
According to the error, it is looking for java 7 installed by cloudera. You should define JAVA_HOME={path_to_your_jdk8_installation} in bashrc.
1 REPLY 1
Highlighted

Re: Unable to launch spark using spark-shell on linux

Expert Contributor
According to the error, it is looking for java 7 installed by cloudera. You should define JAVA_HOME={path_to_your_jdk8_installation} in bashrc.