Reply
New Contributor
Posts: 2
Registered: ‎01-26-2017

CDH install failed with error : Failed to receive heartbeat from agent

I have a small cluster where CDH was installed a month back. Due to some changes, I started experiencing issues with hdfs and hbase, so I decided to cleanup and re-install everything and now I am seeing issues with agent installations, which is failing with error "Failed to receive heartbeat from agent.".

 

I have checked to make sure nodes have proper connectivity and there is no firewall, etc. I also see TCP connections established between server and agent at port 7182 and logs seem to indicate that the agent is healthy, but for some reason the installer fails with the error message.

 

Is this an issue with latest CDH?

 

I tried to reinstall CDH by cleaning all files/directories several times with no luck. I also tried to go back a older version of CDH, but the agent is still being installed with latest version. 

 

bash -c /tmp/scm_prepare_node.B5Mo3Chq/scm_prepare_node.sh --server_version 5.9.1 --server_build 8 --packages

 

Appreciate any help in resolving this.

Posts: 466
Topics: 1
Kudos: 58
Solutions: 38
Registered: ‎08-16-2016

Re: CDH install failed with error : Failed to receive heartbeat from agent

I have seen this when reinstalling the CM Agent without dropping and removing the agent from the cluster and CM.

The reason this happens is that each agent generation a GUID to represent the host and agent. This is unique to each install. CM views the agents still registered to it as the old ones and not the new ones even though it is the same host and it prevents the new ones from registering to CM.

In the CM host page, stop any existing roles, delete them, then deleted the agent. Then the new Host wizard should show the agents now reporting in correctly as unmanaged.
Cloudera Employee
Posts: 41
Registered: ‎10-14-2014

Re: CDH install failed with error : Failed to receive heartbeat from agent

To clarify: is this cluster managed by Cloudera Director, or has this been manually installed?

AFAIK, this is not an issue I've heard of with the most recent CDH.
New Contributor
Posts: 2
Registered: ‎01-26-2017

Re: CDH install failed with error : Failed to receive heartbeat from agent

It seems the issue is related to mulitple hostnames associated with each host. On my setup, I have given more than one hostnames to each host in /etc/hosts. For example, first machine has hostnames "mc1" and "zk-1". I was able to add the hosts successfully when I kept only one hostname for each server.

 

It looks like a bug in CDH as its a common practice to associate multiple hostnames to hosts.

Posts: 466
Topics: 1
Kudos: 58
Solutions: 38
Registered: ‎08-16-2016

Re: CDH install failed with error : Failed to receive heartbeat from agent

I have been able to use multiple aliases with a single host in CDH. Not multiple hostnames though. What did you hosts file look.

It probably is related to how Hadoop does hostname lookups. It used specific Java methods that aren't identical to the hostname CMD, at least across all platforms.
Announcements