sparkR - Error in socketConnection(port = monitorPort)

[#1]

I have centos 7.1.

On my multinode Hadoop cluster (2.3.4) I have , through Ambari, installed spark 1.5.2. I am trying to connect to sparkR from CLI and after I run sparkR I get the following error:

Error in value[[3L]](cond) : Failed to connect JVM In addition: Warning message: In socketConnection(host = hostname, port = port, server = FALSE, : localhost:9001 cannot be opened

The port (9001) is opened on the namenode (where Im running sparkR) Do you have any ideas what Im doing wrong? Ive seen this link: http://hortonworks.com/hadoop-tutorial/apache-spark-1-5-1-technical-preview-with-hdp-2-3/

and I followed also this link:

http://www.jason-french.com/blog/2013/03/11/installing-r-in-linux/

To install R on all datanodes. I appreicate your contribution.

[#2]

@Artem ErvitsI tried that as well.

Here the status command:

sudo service iptables status

And the result:

Redirecting to /bin/systemctl status iptables.service ● iptables.service Loaded: not-found (Reason: No such file or directory) Active: inactive (dead)

I tried running the iptables and firewalld commands on the datanodes and they all seem inactive (same results as above).

Centos 7.2.

[#3]

@marko we don't officially support 7.2 maybe this is one of the reasons? Try to update your host. If you have access to 7.0 or 7.1 machine?

[#4]

@Artem Ervits, @Neeraj Sabharwal Well Here is my version: CentOS Linux release 7.2.1511 (Core) The cloud services I use have, in their menu, CentOS 7.1, I think they upgraded the systems, left 7.1 on the menu but the instances are 7.2

Now im a bit upset. Do you think this could really be it? That 7.2 version is the problem?

[#5]

@marko I see that other people are hitting the same issue. I guess you did the manual install of spark now..Correct?

Can you try to do this in some other node?

[#6]

@Neeraj Sabharwal No, the spark installation was done through Ambari. I installed, accepted all the defaults and ran the test examples from the TP website.Funny thing, when installing, you get to choose 1.4.2 version of spark, while 1.5.2 is installed.

[#7]

@marko Nice!!!!!! You are way ahead 🙂

Can you try running your job from different node?

[#8]

@Neeraj Sabharwal

Nothing nice about this issue, its a show stopper if you work for researchers. 🙂

What do you mean by different node? And run it how?

[#9]

@marko

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_installing_manually_book/content/spark_pr...

[#10]

@Neeraj Sabharwal

Ive manually installed 1.6.0 on the same node where Im running spark 1.5.2. Now I ll try doing it on one datanode - I ll install 1.5.2 manually. By the way, Neeraj, have you seen the CentOS 7.2 dialog Im having with Artem. Let me know if you have a take on that one.

[#11]

@Neeraj Sabharwal

I have now installed 1.5.2 on one of the datanodes, I can run spark-shell, when I run sparkR I get this issue:

Launching java with spark-submit command /usr/hdp/2.3.4.0-3485/spark/bin/spark-submit "sparkr-shell" /tmp/Rtmp6iFIT6/backend_port15af7623f5dd

16/02/17 14:35:18 WARN internal.ThreadLocalRandom: Failed to generate a seed from SecureRandom within 3 seconds. Not enough entrophy?

Error in socketConnection(port = monitorPort) : cannot open the connection

In addition: Warning message: In socketConnection(port = monitorPort) : localhost:44867 cannot be opened

Pretty much the same problem.