Cloudera Community
bhomass1
user rank
Explorer

Re: connection refused on spark client code connec...

by Explorer in Support Questions
‎06-17-2014 07:03 PM
‎06-17-2014 07:03 PM
Actuallty, it is not an issue with firewall. apparently no firewall was set.   I am able to run the same scala code using spark-shell. What I noticed is that when spark-shell connects to workers, it uses all kinds of port numbers but not 7000. inital akka tcp seems to be connecting to spark master on port 55375.   However, when I run the sbt code, it always tries to connect to spark master using 7000. is there a way to set this port number in the scala code.   ... View more

connection refused on spark client code connection...

by Explorer in Support Questions
‎06-16-2014 03:31 PM
‎06-16-2014 03:31 PM
I have a 5 node cluster installed using Cloudera manager 5.0.1.   I can run spark-shell on each of the nodes with no problem. Then I tried connecting using spark client code. I am not able to connect, every node whether master or worker dumps out logs like this.   akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: Connection refused:   each node shows a diffeent port number. When I test telnet to those ports, I always get a connection refused. any one knows how these ports are set, whether fail to telnet means trouble? in general how to approach this problem?   ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎05-07-2015 08:01 PM
Community Statistics
Member Since ‎05-31-2014 10:13 PM
Last Visited ‎05-07-2015 08:01 PM
Posts 11