Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to launch spark using spark-shell on linux

SOLVED Go to solution

Unable to launch spark using spark-shell on linux

Contributor

I am trying to set up SPARK2 on my cloudera cluster. For that, I have JDK1.8:

 

 

I have installed scala 2.11.8 using the rpm file:

I have downloaded, extracted the spark version 2.2.0 on my home directory: /home/cloudera.

 

I made changes to the PATH variable in .bashrc as below:

 

But when I try to execute spark-shell from the home directory: /home/cloudera, it says no such file or directory which can be seen below:

 

[cloudera@quickstart ~]$ spark-shell
/home/cloudera/spark/bin/spark-class: line 71: /usr/java/jdk1.7.0_67-cloudera/bin/java: No such file or directory
[cloudera@quickstart ~]$

 

Could anyone let me know how can I fix the problem and configure it properly ?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Unable to launch spark using spark-shell on linux

Expert Contributor
According to the error, it is looking for java 7 installed by cloudera. You should define JAVA_HOME={path_to_your_jdk8_installation} in bashrc.
1 REPLY 1

Re: Unable to launch spark using spark-shell on linux

Expert Contributor
According to the error, it is looking for java 7 installed by cloudera. You should define JAVA_HOME={path_to_your_jdk8_installation} in bashrc.