Support Questions
Find answers, ask questions, and share your expertise

SparkR - Error in socketConnection(port = monitorPort) in centos 6.6

SparkR - Error in socketConnection(port = monitorPort) in centos 6.6

Explorer

I installed HDP 2.3.4 on mutinodes , through blueprint using Ambari, installed spark 1.5.2.

and install R 3.3.0.

When I run sparkR on one node, it show the error message. Following

Error in socketConnection(port = monitorPort) : cannot open the connection In addition: Warning message: In socketConnection(port = monitorPort) : localhost:32001 cannot be opened

so, I also follow https://community.hortonworks.com/articles/21304/installing-spark-16-on-hdp-23x.html to install spark 1.6

and run sparkR.

But, it also show the same error message.

7 REPLIES 7

Re: SparkR - Error in socketConnection(port = monitorPort) in centos 6.6

Please verify /etc/hosts entry of particular node from where you running sparkR and try changing the localhost to local IP address.

Re: SparkR - Error in socketConnection(port = monitorPort) in centos 6.6

Explorer

I change /etc/hosts. As follow:

127.0.1.1 hostname

127.0.0.1 localhost

It can fix this problem. But,I remember that adding 127.0.1.1 to hosts has some side effect in Centos.

Re: SparkR - Error in socketConnection(port = monitorPort) in centos 6.6

what are the side effects? could you please explain.

Re: SparkR - Error in socketConnection(port = monitorPort) in centos 6.6

Explorer

I just do this, but I don't know why i can works.

Re: SparkR - Error in socketConnection(port = monitorPort) in centos 6.6

Explorer

I found the root cause, thanks.

Re: SparkR - Error in socketConnection(port = monitorPort) in centos 6.6

Explorer

I know the root cause.

The following is resolv.conf:

nameserver 192.168.136.254 ;

generated by /sbin/dhclient-script

nameserver 192.168.122.1

But, "nameserver 192.168.122.1" is unused.

SparkR works by deleting "nameserver 192.168.122.1" from resolv.conf

Re: SparkR - Error in socketConnection(port = monitorPort) in centos 6.6

Explorer

/etc/nsswitch.conf

hosts: dns files

So, dns server hanging.

SparkR job socket timeout because wait for dns answer.