sparkR - Error in socketConnection(port = monitorPort)

[#1]

I have centos 7.1.

On my multinode Hadoop cluster (2.3.4) I have , through Ambari, installed spark 1.5.2. I am trying to connect to sparkR from CLI and after I run sparkR I get the following error:

Error in value[[3L]](cond) : Failed to connect JVM In addition: Warning message: In socketConnection(host = hostname, port = port, server = FALSE, : localhost:9001 cannot be opened

The port (9001) is opened on the namenode (where Im running sparkR) Do you have any ideas what Im doing wrong? Ive seen this link: http://hortonworks.com/hadoop-tutorial/apache-spark-1-5-1-technical-preview-with-hdp-2-3/

and I followed also this link:

http://www.jason-french.com/blog/2013/03/11/installing-r-in-linux/

To install R on all datanodes. I appreicate your contribution.

[#2]

@marko

please use the steps here http://hortonworks.com/hadoop-tutorial/apache-spark-1-6-technical-preview-with-hdp-2-3/

[#3]

@marko check whether firewall is blocking the port on each node.

[#4]

@Artem Ervits

Ive seen this one as well, dont see a big difference between this one and 1.5.2

I have SPARK_HOME and JAVA_HOME defined.

My hive-site.xml is also on its place.

If I scroll down to the SparkR part: R is installed on all the nodes.

by the way, when I run sparkR, I dont get the nice Spark graphic (logo) seems as if Im starting just R.

[#5]

@marko

I recommend this http://hortonworks.com/hadoop/spark/#section_6 but your link is good too based on your spark version.

[#6]

@marko Now, let's troubleshoot 9001 issue.

netstat -anp | grep 9001 --> whats the output?

[#7]

@Neeraj Sabharwal

running sudo netstat -anp | grep 9001

returns:

unix 2 [ ACC ] STREAM LISTENING 9001 1202/master private/proxywrite

[#8]

@marko ps -ef | grep 1202

If you don't need it then kill it ...

[#9]

@Neeraj Sabharwal

I killed process, also restarted spark from ambari.

If I run sudo netstat -anp | grep 9001 I dont see anything.

I also have this one in my bashrc on the node where Im running sparkR:

export EXISTING_SPARKR_BACKEND_PORT=9001

Funny thing, if i run sparkR with my centos user I get the error mentioned in the original post.

If i run sudo -u spark sparkR then I get:

Error in socketConnection(port = monitorPort) : cannot open the connection In addition: Warning message: In socketConnection(port = monitorPort) : localhost:53654 cannot be opened

[#10]

@marko Interesting...see this

http://pastebin.com/VefGqMea