Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Connecting to spark master in quickstart VM via spark shell fails from a different node

Connecting to spark master in quickstart VM via spark shell fails from a different node

New Contributor

Hi,

I am trying to connect a shell to a spark master running in a quickstart VM (5.4.2). Locally it works fine but from a different node (also a quickstart VM) I am getting no conneciton. I am trying to connect with the following command: /usr/bin/spark-shell --master spark://192.168.1.31:7077. With nmap I see that the port is open but I am getting the following from the spark-shell:

 

INFO client.AppClient$ClientActor: Connecting to master akka.tcp://sparkMaster@192.168.1.31:7077/user/Master...
ERROR cluster.SparkDeploySchedulerBackend: Application has been killed. Reason: All masters are unresponsive! Giving up.

 

Anyone an idea what the issue might be? Is there any particularity of the quickstart VM that prevents this connection?

1 REPLY 1

Re: Connecting to spark master in quickstart VM via spark shell fails from a different node

Master Guru
Does it work if you connect to it locally from the same VM? If yes, then you may have a networking or firewall misconfiguration perhaps. I'd recommend running "python -m SimpleHTTPServer -p 7077" on both sides and try to load them in each other's browsers, to ascertain if a full connectivity is present and functional between the two VMs.