Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

When adding New Host: Installation failed. Failed to receive heartbeat from agent (yes, again)

When adding New Host: Installation failed. Failed to receive heartbeat from agent (yes, again)

Explorer

Centos 6.7.  CM 5.7

 

I reviewed the similar posts on this topic but the suggestions did not work.  I am adding a node to the cluster through "Add New Hosts to a Cluster".  The install progressed but ultimatly failed for heartbeat to the Cloudera Manager node.

 

Failure Text: (bold is my findings)

Installation failed. Failed to receive heartbeat from agent.

  • Ensure that the host's hostname is configured properly. (all hostname return the correct value)
  • Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules). (listening)
  • Ensure that ports 9000 and 9001 are not in use on the host being added. (unused and open)
  • Check agent logs in /var/log/cloudera-scm-agent/ on the host being added. (Some of the logs can be found in the installation details). (see below)
  • If Use TLS Encryption for Agentsis enabled in Cloudera Manager (Administration -> Settings -> Security), ensure that /etc/cloudera-scm-agent/config.ini has use_tls=1 on the host being added. Restart the corresponding agent and click the Retry link here. (setting is 1)

Here is the last line of  cloudera-scm-agent.log.  

 

[16/Nov/2016 11:40:51 +0000] 9582 MainThread agent        ERROR    Heartbeating to dcl-hdp-100-en1.choa.org:7182 failed.
Traceback (most recent call last):
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.7.0-py2.7.egg/cmf/agent.py", line 1188, in _send_heartbeat
    self.max_cert_depth)
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.7.0-py2.7.egg/cmf/https.py", line 131, in __init__
    self.conn.connect()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/M2Crypto-0.21.1-py2.7-linux-x86_64.egg/M2Crypto/httpslib.py", line 50, in connect
    self.sock.connect((self.host, self.port))
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/M2Crypto-0.21.1-py2.7-linux-x86_64.egg/M2Crypto/SSL/Connection.py", line 185, in connect
    ret = self.connect_ssl()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/M2Crypto-0.21.1-py2.7-linux-x86_64.egg/M2Crypto/SSL/Connection.py", line 178, in connect_ssl
    return m2.ssl_connect(self.ssl)
SSLError: unexpected eof

Please let me know your thoughts or other logs you may need to help.

 

Thanks, Tod

2 REPLIES 2

Re: When adding New Host: Installation failed. Failed to receive heartbeat from agent (yes, again)

Super Guru

Hi,

 

My first guess is that you do not have the agent configured for the level of security that is set in Cloudera Manager for agent communication.  I recommend checking a working agent's /etc/cloudera-scm-agent/config.ini and verify that all the security settings are the same.  In fact, the config.ini should be the same on all nodes.

 

Regards,

 

Ben

Re: When adding New Host: Installation failed. Failed to receive heartbeat from agent (yes, again)

Explorer

Thanks for the response.  The only difference is that the target host has no values for the two properties below.

 

Is copying those the right next step ?

 

# PEM file containing client private key.
client_key_file=/opt/cloudera/.....

# PEM file containing client certificate.
client_cert_file=/opt/cloudera/.....

Don't have an account?
Coming from Hortonworks? Activate your account here