question Re: CM 7.13.1 Installation failed. Failed to receive heartbeat from agent. AvroTypeException in Support Questions

CM 7.13.1 Installation failed. Failed to receive heartbeat from agent. AvroTypeException

AgustinG — Tue, 11 Feb 2025 10:18:48 GMT

Hello,
I'm facing some issues trying to install CM 7.13.1 on Ubuntu containers (happens in both 22.04 and 20.04) using a Postgres 10 14 server running on its own container.

The install of both CM server and CM agents work fine, but when I start the cluster creation it fails during the CM agents startup. The following error is reported in the agent logs (cloudera-scm-agent.log):

[11/Feb/2025 09:32:33 +0000] 8471 MainThread agent ERROR Heartbeating to node-1.cluster:7182 failed. Traceback (most recent call last): File "/opt/cloudera/cm-agent/lib/python3.8/site-packages/cmf/agent.py", line 1642, in _send_heartbeat response = self.requestor.request('heartbeat', heartbeat_data) File "/opt/cloudera/cm-agent/lib/python3.8/site-packages/avro/ipc.py", line 132, in request self.write_call_request(message_name, request_datum, buffer_encoder) File "/opt/cloudera/cm-agent/lib/python3.8/site-packages/avro/ipc.py", line 174, in write_call_request self.write_request(message.request, request_datum, encoder) File "/opt/cloudera/cm-agent/lib/python3.8/site-packages/avro/ipc.py", line 178, in write_request datum_writer.write(request_datum, encoder) File "/opt/cloudera/cm-agent/lib/python3.8/site-packages/avro/io.py", line 979, in write raise AvroTypeException(self.writers_schema, datum) avro.io.AvroTypeException: The datum ... formatted python dictionary .... is not an example of the schema [... whole avro schema...]

This error looks similar to a previous bug reported here , although this is reported as solved. I've also tried installing CM 7.11.3 on the same Ubuntu 20.04 images, just to check whether this might not be something related to installing CM in a container, but the installation was successful in this case and the agents managed to connect with the CM server.

Any ideas on what I can try here to get this working?

Thanks!

Edit: I had reported a non supported Postgres Version, but the issue persists when upgrading it to Postgres 14.

Edit2: Adding here that i'm using OpenJDK 17

Re: CM 7.13.1 Installation failed. Failed to receive heartbeat from agent. AvroTypeException

VidyaSargur — Tue, 11 Feb 2025 09:57:21 GMT

@AgustinG, Welcome to our community! To help you get the best possible answer, I have tagged our Cloudera Manager experts @paras @Gopinath who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Re: CM 7.13.1 Installation failed. Failed to receive heartbeat from agent. AvroTypeException

AgustinG — Fri, 14 Feb 2025 09:11:47 GMT

It appears that the issue is not appearing when using docker (I was using podman before).

I'm not sure how it could affect the comms with the agents, but I've had issues on previous versions related to the DNS resolution of the nodes returning multiple names that do not necessarily correspond to the hostname configured in /etc/hostname

I will try to confirm this and update this thread for anyone trying the same, but I'll mark this as solved since I can confirm the issue is related to using podman.