Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

OS upgrade and lost heartbeat

Highlighted

OS upgrade and lost heartbeat

New Contributor

I work on a small dev cluster (4 machines) running on HDP 3.0.1 and recently lost all heartbeat on the master node due to an unplanned upgrade of my OS (ubuntu16 to 18). I still don't know why or how this node got its OS upgraded but I can't rollback to the previous version.

The log from ambari-server reads:

2019-05-20 17:06:50,553  INFO [agent-register-processor-7] HeartBeatHandler:317 - agentOsType = ubuntu18
2019-05-20 17:06:50,554  WARN [agent-register-processor-7] HeartBeatHandler:319 - Received registration request from host with not supported os type, hostname=hadoopmaster.*****.***, serverOsType=ubuntu16, agentOsType=ubuntu18
2019-05-20 17:06:50,554  INFO [agent-register-processor-7] HeartbeatController:105 - Cannot register host with not supported os type, hostname=hadoopmaster.*****.***, serverOsType=ubuntu16, agentOsType=ubuntu18
org.apache.ambari.server.AmbariException: Cannot register host with not supported os type, hostname=hadoopmaster.*****.***, serverOsType=ubuntu16, agentOsType=ubuntu18
    at org.apache.ambari.server.agent.HeartBeatHandler.handleRegistration(HeartBeatHandler.java:327)
    at org.apache.ambari.server.agent.stomp.HeartbeatController.lambda$register$0(HeartbeatController.java:100)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)


I stopped all services using ambari interface and tried to use the "Recover host" option but I get the following message:

108803-1558365270780.png

I'm not sure how to interpret this error. Are the services all stopped but ambari is unable to get that information or are there some services still running on the master node that I need to turn off manually? The master node has been rebooted since the OS upgrade and there is no script to automatically start ambari services so they should all be stopped.Would a host recovery even work in the case of an OS upgrade?


I read that the last ambari version supports Ubuntu18. Would an upgrade fix the problem?

Thank you for your help and advices.

2 REPLIES 2

Re: OS upgrade and lost heartbeat

Super Mentor

@Jeremy Jean-Jean

Please make sure that your Ambari Server and Agent versions are atleast 2.7.3 if you want to use "ubuntu-18".



Please refer to the following link and then click on the ambari version tat you are using to findout if the OS that you are using is supported or not: https://supportmatrix.hortonworks.com/

.


Re: OS upgrade and lost heartbeat

New Contributor

Thank you for your answer. The OS upgrade (Ubuntu16 - Ubuntu18) wasn't planned and so my current ambari version is 2.7.0 which does not support Ubuntu18.

Do you think it is possible to upgrade ambari 2.7.3 and solve the heartbeat problem or am I forced to reinstall the whole cluster?