Reply
Explorer
Posts: 15
Registered: ‎11-27-2017

Failed to add host

[ Edited ]

I'm trying to add a host to a single node cluster after configuring it with tls/ssl and kerberos but am unable to.  What should I check?

 

 Installation failed. Failed to receive heartbeat from agent.

  • Ensure that the host's hostname is configured properly.
  • Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules).
  • Ensure that ports 9000 and 9001 are not in use on the host being added.
  • Check agent logs in /var/log/cloudera-scm-agent/ on the host being added. (Some of the logs can be found in the installation details).
  • If Use TLS Encryption for Agents is enabled in Cloudera Manager (Administration -> Settings -> Security), ensure that /etc/cloudera-scm-agent/config.ini has use_tls=1 on the host being added. Restart the corresponding agent and click the Retry link here.

 

>>SSLError: unexpected eof
>>[25/May/2018 11:43:31 +0000] 11441 MainThread agent ERROR Heartbeating to 192.168.0.11:7182 failed.
>>Traceback (most recent call last):
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.14.3-py2.7.egg/cmf/agent.py", line 1424, in _send_heartbeat
>> self.max_cert_depth)
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.14.3-py2.7.egg/cmf/https.py", line 138, in __init__
>> self.conn.connect()
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/M2Crypto-0.24.0-py2.7-linux-x86_64.egg/M2Crypto/httpslib.py", line 59, in connect
>> sock.connect((self.host, self.port))
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/M2Crypto-0.24.0-py2.7-linux-x86_64.egg/M2Crypto/SSL/Connection.py", line 195, in connect
>> ret = self.connect_ssl()
>> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/M2Crypto-0.24.0-py2.7-linux-x86_64.egg/M2Crypto/SSL/Connection.py", line 188, in connect_ssl
>> return m2.ssl_connect(self.ssl, self._timeout)

>>SSLError: unexpected eof
>>[25/May/2018 11:43:32 +0000] 11441 MainThread agent INFO Stopping agent...
>>[25/May/2018 11:43:32 +0000] 11441 MainThread agent INFO No extant cgroups; unmounting any cgroup roots
>>[25/May/2018 11:43:32 +0000] 11441 MainThread agent INFO 1 processes are being managed; Supervisor will continue to run.
>>[25/May/2018 11:43:32 +0000] 11441 MainThread _cplogging INFO [25/May/2018:11:43:32] ENGINE Bus STOPPING
>>[25/May/2018 11:43:32 +0000] 11441 MainThread _cplogging INFO [25/May/2018:11:43:32] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('phd-node1', 9000)) shut down
>>[25/May/2018 11:43:32 +0000] 11441 MainThread _cplogging INFO [25/May/2018:11:43:32] ENGINE Stopped thread '_TimeoutMonitor'.
>>[25/May/2018 11:43:32 +0000] 11441 MainThread _cplogging INFO [25/May/2018:11:43:32] ENGINE Bus STOPPED
>>[25/May/2018 11:43:32 +0000] 11441 MainThread _cplogging INFO [25/May/2018:11:43:32] ENGINE Bus STOPPING
>>[25/May/2018 11:43:32 +0000] 11441 MainThread _cplogging INFO [25/May/2018:11:43:32] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('phd-node1', 9000)) already shut down
>>[25/May/2018 11:43:32 +0000] 11441 MainThread _cplogging INFO [25/May/2018:11:43:32] ENGINE No thread running for None.
>>[25/May/2018 11:43:32 +0000] 11441 MainThread _cplogging INFO [25/May/2018:11:43:32] ENGINE Bus STOPPED
>>[25/May/2018 11:43:32 +0000] 11441 MainThread _cplogging INFO [25/May/2018:11:43:32] ENGINE Bus EXITING
>>[25/May/2018 11:43:32 +0000] 11441 MainThread _cplogging INFO [25/May/2018:11:43:32] ENGINE Bus EXITED
>>[25/May/2018 11:43:32 +0000] 11441 MainThread agent INFO Agent exiting; caught signal 15
>>[25/May/2018 11:43:32 +0000] 11441 Dummy-13 daemonize WARNING Stopping daemon.
END (0)
end of agent logs.
scm agent started
Installation script completed successfully.

Expert Contributor
Posts: 127
Registered: ‎01-08-2018

Re: Failed to add host

You can check the last of the points you have listed as it is almost sure it will fail to hearbeat, if you have not manually did that step (set use_tls=1 and restart agent)

If Use TLS Encryption for Agents is enabled in Cloudera Manager (Administration -> Settings -> Security), ensure that /etc/cloudera-scm-agent/config.ini has use_tls=1 on the host being added. Restart the corresponding agent and click the Retry link here.

If you did this already, then make sure that the configured keystore and truststore files, have been copied to the new host.

Explorer
Posts: 15
Registered: ‎11-27-2017

Re: Failed to add host

Hi Gekas, thanks for the reply. I copied the config.ini from the head node to the compute node to make sure all the config items are the same.  I've verified use_tls=1.  Anything else that might cause the additional node to not heartbeat?

Highlighted
Expert Contributor
Posts: 127
Registered: ‎01-08-2018

Re: Failed to add host

Have you checked the trust store file that it is copied to the host?
Explorer
Posts: 15
Registered: ‎11-27-2017

Re: Failed to add host

I copied $JAVA_HOME/jre/lib/security/jssecacerts from the head node to the compute node.  What should I check for?  How do I check the trust store?

Announcements