question Re: Cloudera Management service failed to start after reboot in Support Questions

Cloudera Management service failed to start after reboot

sharathkumar13 — Thu, 21 Jan 2021 11:35:00 GMT

Hi,

I'm not able to start the management service after the reboot of the manager node. I'm not sure what happened, everything was working fine until the reboot.

[21/Jan/2021 11:03:25 +0000] 15063 MainThread agent ERROR Heartbeating to ip-172-31-203-46.ec2.internal:7182 failed.
Traceback (most recent call last):
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/agent.py", line 1399, in _send_heartbeat
response = self.requestor.request('heartbeat', heartbeat_data)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 141, in request
return self.issue_request(call_request, message_name, request_datum)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 254, in issue_request
call_response = self.transceiver.transceive(call_request)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 483, in transceive
result = self.read_framed_message()
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 489, in read_framed_message
framed_message = response_reader.read_framed_message()
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 417, in read_framed_message
raise ConnectionClosedException("Reader read 0 bytes.")
ConnectionClosedException: Reader read 0 bytes.

The error message in UI is

"Role not started due to unhealthy host"

Re: Cloudera Management service failed to start after reboot

GangWar — Mon, 25 Jan 2021 11:46:53 GMT

@sharathkumar13 The error indicates that your hosts are not heartbeating to Cloudera Manager hence you are not able to start mgmt service.

Please check the config.ini file and see if the CM server hosts name is correct and also check if agent is running on the host.

Re: Cloudera Management service failed to start after reboot

sharathkumar13 — Mon, 25 Jan 2021 17:36:56 GMT

Thanks for the reply, @GangWar.

The issue is sorted out now. As you said, the issue got fixed after checking the config.ini. The inter communications between the client and server were made to do over TLS, but we disabled the TLS/SSL in UI and for all agents. This was the main cause, as the heartbeat tried to get a response, but it didnt. Changing the configuration in Cloudera Manager -> Administration -> “Use TLS Encryption for Agents” should be unchecked, and restarting the Cloudera Manager fixed the issue.