Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cannot connect to ResoureManager

Solved Go to solution
Highlighted

Cannot connect to ResoureManager

Contributor

Since HiveUI is not working through Ambari, on troubleshooting through "hive -hiveconf hive.root.logger=DEBUG,console",

16/04/13 06:44:37 [IPC Client (1720891078) connection to RM_FQDN /RM_IP:8020 from root]: DEBUG ipc.Client: IPC Client (1720891078) connection to RM_FQDN/RM_IP:8020 from root got value #32

16/04/13 06:44:37 [main]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms

16/04/13 06:44:37 [main]: DEBUG ipc.Client: The ping interval is 60000 ms.

16/04/13 06:44:37 [main]: DEBUG ipc.Client: Connecting to RM_FQDN/RM_IP:8050

16/04/13 06:44:38 [main]: INFO ipc.Client: Retrying connect to server: RM_FQDN/RM_IP:8050. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)

16/04/13 06:44:39 [main]: INFO ipc.Client: Retrying connect to server: RM_FQDN/RM_IP:8050. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)

Also ResourceManager UI giving "ERR_CONNECTION_REFUSED"

Seems even when RM is already started through Ambari, it cannot be accessed/connected through client or UI.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Cannot connect to ResoureManager

Contributor

Here's how I resolved the issue:

RM log was showing some issue related to ZK

:2016-04-18 05:20:57,275 INFO recovery.ZKRMStateStore (ZKRMStateStore.java:runWithRetries(1214)) - Retrying operation on ZK. Retry no. 195 2016-04-18 05:20:57,454 INFO zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(1019)) - Opening socket connection to server ZK_HOST/10.131.$ 2016-04-18 05:20:57,455 WARN zookeeper.ClientCnxn (ClientCnxn.java:run(1146)) - Session 0x0 for server null, unexpected error, closing socket connection and attemptin$ java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)

Also noticed ZK kept getting stopped. ZK logs: java.io.FileNotFoundException: /hadoop/zookeeper/version-2/log. (Permission denied)

So did chmod a+rwx -R /hadoop/zookeeper andd then restarted ZK and RM services.

After that was able to successfully start Hive CLI.

Thanks @Kuldeep Kulkarni @Laurent Edel

View solution in original post

7 REPLIES 7
Highlighted

Re: Cannot connect to ResoureManager

can you please check the RM logs for any possible errors?

Highlighted

Re: Cannot connect to ResoureManager

Guru

ResourceManager should not be on port 8020, default is 8088. 8020 is used by NameNode.

Highlighted

Re: Cannot connect to ResoureManager

Super Guru
@Amit Tewari

1. Can you please check if resource manager is listening on 8050

e.g.

[root@sandbox coord]# netstat -tulpn|grep 8050
tcp        0      0 0.0.0.0:8050                0.0.0.0:*                   LISTEN      8102/jav

2. If yes, can you please check if is reachable from hive client ? you can try simple ping command first and if ping is working then try to do telnet from hive client

e.g.

[root@sandbox coord]# telnet sandbox.hortonworks.com 8050
Trying 10.0.2.15...
Connected to sandbox.hortonworks.com.
Escape character is '^]'.

3. If you still get connection refused then try telnet from some other node in the cluster, if it works then your hive client has some connectivity issues with RM

4. If it doesn't work from any of the host then RM has some issues, you might need to check RM logs or RM host if any firewall is blocking packets to 8050.

Hope this helps.

Highlighted

Re: Cannot connect to ResoureManager

Contributor
@Kuldeep Kulkarni Couldnt connect using telnet
Highlighted

Re: Cannot connect to ResoureManager

Super Guru
@Amit Tewari

Is netstat working? please paste output of netstat command here

Highlighted

Re: Cannot connect to ResoureManager

Super Guru

@Amit Tewari - Please check my answer below and accept if this resolves your issue

Highlighted

Re: Cannot connect to ResoureManager

Contributor

Here's how I resolved the issue:

RM log was showing some issue related to ZK

:2016-04-18 05:20:57,275 INFO recovery.ZKRMStateStore (ZKRMStateStore.java:runWithRetries(1214)) - Retrying operation on ZK. Retry no. 195 2016-04-18 05:20:57,454 INFO zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(1019)) - Opening socket connection to server ZK_HOST/10.131.$ 2016-04-18 05:20:57,455 WARN zookeeper.ClientCnxn (ClientCnxn.java:run(1146)) - Session 0x0 for server null, unexpected error, closing socket connection and attemptin$ java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)

Also noticed ZK kept getting stopped. ZK logs: java.io.FileNotFoundException: /hadoop/zookeeper/version-2/log. (Permission denied)

So did chmod a+rwx -R /hadoop/zookeeper andd then restarted ZK and RM services.

After that was able to successfully start Hive CLI.

Thanks @Kuldeep Kulkarni @Laurent Edel

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here