Support Questions

Find answers, ask questions, and share your expertise

Spark -Shell does not start

avatar
New Contributor

When I try to start spark-shell using Cloudera quick start VM 5.8 and 5.10 it just hangs and does not do anything,

is it any solution available for this.I have attached the screen shot. Any help ?screen shot.PNG

 Spark services are running

7 REPLIES 7

avatar
Champion

For some strange reason I cant open ur screenshot .

could you paste the error in the Insert code tag. 

 

Could you let me know if you had set your Java_home path  and Spark_home ? 

master and worker node status ?

what type of spark are you trying 

 

  • Standalone - meaning Spark will manage its own cluster
  • YARN - using Hadoop's YARN resource manager
  • Mesos - 

try this command  -  in your command line prompt 

SPARK_HOME/bin/pyspark

or

SPARK_HOME/bin/spark-shell

 

 

 

avatar
New Contributor

Hi,

 

Thanks for the response. I am using cloudera quick start VM 5.10 with 16 GB RAM.I tried the below path

 

/usr/lib/spark>spark-shell

 

 

However all it says 

 

Service SparkUI could not bind on port 4040.Attempting port 4041

 

and hangs...means does not proceed further

 

 

avatar
Expert Contributor
how did you solve this?

avatar
New Contributor

This weekend we encounter the same issue.

 

we changed the resource manager to standby on the current node and made active on another node.

spark-shell is working.

 

Still problem exists on original node. Unable to identify the issue

avatar
Community Manager

@sridhar-exelon as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
New Contributor

Thank you Vidya

avatar
Expert Contributor

I think in my case the issue was with multiple version of java. I dont remember exactly its been a while