I installed HDP 2.3.4 on mutinodes , through blueprint using Ambari, installed spark 1.5.2.
and install R 3.3.0.
When I run sparkR on one node, it show the error message. Following
Error in socketConnection(port = monitorPort) : cannot open the connection In addition: Warning message: In socketConnection(port = monitorPort) : localhost:32001 cannot be opened
so, I also follow https://community.hortonworks.com/articles/21304/installing-spark-16-on-hdp-23x.html to install spark 1.6
and run sparkR.
But, it also show the same error message.
Please verify /etc/hosts entry of particular node from where you running sparkR and try changing the localhost to local IP address.
I change /etc/hosts. As follow:
It can fix this problem. But,I remember that adding 127.0.1.1 to hosts has some side effect in Centos.
I know the root cause.
The following is resolv.conf:
nameserver 192.168.136.254 ;
generated by /sbin/dhclient-script
But, "nameserver 192.168.122.1" is unused.
SparkR works by deleting "nameserver 192.168.122.1" from resolv.conf
hosts: dns files
So, dns server hanging.
SparkR job socket timeout because wait for dns answer.