Support Questions

Find answers, ask questions, and share your expertise

adding host in cluster giving no route to host

avatar
Explorer

i am trying to add a new host in cluster but its hanging at parcel downloading .

 

checked cloudera-scm-server.log .

 

2014-12-30 22:52:04,535 INFO 983864240@scm-web-13:com.cloudera.server.cmf.node.NodeScannerService: Request 0 returning 0/1 scans
2014-12-30 22:52:04,564 INFO NodeScannerThread-0:net.schmizz.sshj.transport.TransportImpl: Client identity string: SSH-2.0-SSHJ_0_8
2014-12-30 22:52:04,570 INFO NodeScannerThread-0:net.schmizz.sshj.transport.TransportImpl: Server identity string: SSH-2.0-OpenSSH_5.3
2014-12-30 22:52:04,571 INFO NodeScannerThread-0:net.schmizz.sshj.transport.KeyExchanger: Sending SSH_MSG_KEXINIT
2014-12-30 22:52:04,586 INFO reader:net.schmizz.sshj.transport.KeyExchanger: Received SSH_MSG_KEXINIT
2014-12-30 22:52:04,696 INFO reader:net.schmizz.sshj.transport.kex.DHG14: Sending SSH_MSG_KEXDH_INIT
2014-12-30 22:52:04,699 INFO reader:net.schmizz.sshj.transport.KeyExchanger: Received kex followup data
2014-12-30 22:52:04,699 INFO reader:net.schmizz.sshj.transport.kex.DHG14: Received SSH_MSG_KEXDH_REPLY
2014-12-30 22:52:04,815 INFO reader:net.schmizz.sshj.transport.KeyExchanger: Sending SSH_MSG_NEWKEYS
2014-12-30 22:52:04,815 INFO reader:net.schmizz.sshj.transport.KeyExchanger: Received SSH_MSG_NEWKEYS
2014-12-30 22:52:04,837 INFO NodeScannerThread-0:com.cloudera.server.cmf.node.CmfSSHClient: Key exchange took 0.266 seconds
2014-12-30 22:52:04,837 INFO NodeScannerThread-0:com.cloudera.server.cmf.node.NodeScanner: Disconnecting from ssh service on remote host
2014-12-30 22:52:04,844 INFO NodeScannerThread-0:com.cloudera.server.cmf.node.NodeScanner: Connected to SSH on node rac1 with port 22 (latency PT0.007S)
2014-12-30 22:52:04,844 INFO NodeScannerThread-0:com.cloudera.server.cmf.node.NodeScannerService: Request 0 observed finished scan of node rac1
2014-12-30 22:52:05,545 INFO 983864240@scm-web-13:com.cloudera.server.cmf.node.NodeScannerService: Request 0 returning 1/1 scans
2014-12-30 22:52:19,165 INFO 1857968490@scm-web-12:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command ProcessStalenessCheckCommand BasicCmdArgs{args=[First reason why: com.cloudera.cmf.model.DbHost.cluster (#7) has changed]}.
2014-12-30 22:52:19,219 WARN New I/O client boss #1-0:com.cloudera.server.cmf.HeartbeatRequester: Error requesting heartbeat of host id 768dab4e-fc30-4915-b420-154bcb293652
java.net.ConnectException: No route to host to http://rac1:9000/heartbeat
        at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:100)
        at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:399)
        at org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:390)
        at org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:352)
        at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:409)
        at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:366)
        at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:282)
        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
        at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.NoRouteToHostException: No route to host

        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:404)
        ... 7 more
2014-12-30 22:52:21,077 WARN 1857968490@scm-web-12:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:52:28,098 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check done. Duration: PT8.858S
2014-12-30 22:52:35,684 WARN 296858772@scm-web-18:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:52:50,734 WARN 983864240@scm-web-13:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:53:05,766 WARN 983864240@scm-web-13:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:53:20,758 WARN 983864240@scm-web-13:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:53:35,787 WARN 983864240@scm-web-13:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:53:50,797 WARN 1857968490@scm-web-12:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:54:05,851 WARN 1857968490@scm-web-12:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:54:20,812 WARN 1857968490@scm-web-12:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:54:35,849 WARN 1857968490@scm-web-12:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:54:50,846 WARN 49742403@scm-web-16:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:55:05,857 WARN 49742403@scm-web-16:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:55:20,854 WARN 49742403@scm-web-16:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:55:35,879 WARN 49742403@scm-web-16:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:55:50,867 WARN 396202713@scm-web-17:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel
2014-12-30 22:56:05,951 WARN 396202713@scm-web-17:com.cloudera.parcel.components.LocalParcelManagerImpl: Unable to find parcel in the file system: CDH-5.3.0-1.cdh5.3.0.p0.30-el6.parcel

 

 

can could be the reason for this. however i am able to access this host (using ssh) from master node .

 

1 ACCEPTED SOLUTION

avatar
Please ensure naming is correctly set up. Both the new node and the host running CM should be able to resolve each other's host name.

If you can resolve host name correctly but cannot connect to ports other than the standard ones like ssh, then ensure that the firewall (iptables) on the host is turned off
Regards,
Gautam Gopalakrishnan

View solution in original post

4 REPLIES 4

avatar
Please ensure naming is correctly set up. Both the new node and the host running CM should be able to resolve each other's host name.

If you can resolve host name correctly but cannot connect to ports other than the standard ones like ssh, then ensure that the firewall (iptables) on the host is turned off
Regards,
Gautam Gopalakrishnan

avatar
Explorer

Thanks I resolved it .

avatar
New Contributor
I am doing everything as you mentioned in this post. I still get the same error. Could there be any reason?

avatar
New Contributor

It could be the issue with firewall 

Solution: 

 

Stop the firewall: > service iptables stop

allow the port you want to connect on the host: > ufw allow [port_number]