Support Questions

Find answers, ask questions, and share your expertise

All of ambari components are failing to start, manual starts do not work.

avatar
Explorer

Everytime I manually start it either loses hearbeat after a few components starting or all components still fail to start. When I try to start all components with the one click option, it never finishes. When I try to restart all components, it never finishes. Can someone guide me to fix this problem? I am trying to upgrade from hdp 2.3.2 to hdp 2.4. I have already upgraded ambari and only need all ambari components to start to upgrade hdp

1 ACCEPTED SOLUTION

avatar
Explorer

I think I must have not installed properly when I first installed hdp 2.3.2. I say this because I have completely deleted hdp 2.3.2 from the Virtualbox and installed hdp 2.4 directly. In doing this, ambari is working perfectly fine with no errors or alerts unlike previously.

View solution in original post

4 REPLIES 4

avatar
Master Mentor

Please follow all prerequisites in the documentation, also post retries you get, your question is too generic.

avatar
Master Guru

Have you followed Ambari Upgrade guide, and could you complete all steps without errrors? Ambari upgrade is easy, provided that you had started with a healthy cluster. That's why the very first step of every Ambari upgrade is to inspect the cluster, make sure that all components are running and clear all alerts and warnings. After that back-up all cluster supporting data-bases, and follow these steps:

  • Stop Ambari server and Ambari agents on all hosts
  • Prepare and distribute a new version of ambari.repo to all cluster nodes
  • Upgrade Ambari server (on CentOS/RHEL run yum upgrade ambari-server), confirm it was successful, see Step 6 in the above link.
  • Upgrade Ambari agents on all nodes (yum upgrade ambari-agent), confirm it was successful, see Step 8 in the above link.
  • Upgrade Ambari data-base, run "ambari-server upgrade"
  • "ambari-server start"
  • "ambari-agent start" on all nodes in the cluster

All above steps were supposed to complete without errors. Was that so? What you can do now:

  • Repeat Steps 6 and 8 from the guide to make sure you have correct versions of Ambari server and agents
  • Make sure Ambari server is running: ambari-server status
  • Make sure all Ambari agents are running: run "ambari-agent status" on all nodes
  • Open Ambari web UI, if you still have hearbeat lost question marks, then something is wrong with Ambari agents and/or server. Inspect logs for errors: /var/log/ambari-server and /var/log/ambari-agent.

Once Ambari is running you can begin starting services: start them one by one, ZooKeeper first, then HDFS, then all the others. Inspect Ambari tasks, and drill down for errors if they fail (red markings). Hope this helps.

avatar
Explorer

I think I must have not installed properly when I first installed hdp 2.3.2. I say this because I have completely deleted hdp 2.3.2 from the Virtualbox and installed hdp 2.4 directly. In doing this, ambari is working perfectly fine with no errors or alerts unlike previously.

avatar
Expert Contributor

I struggled through this for longer than a while and then decided to share it here:

The trick is to basically delete all the symlinks pointing back to locations where HDP components reside since that is what causes 80% of the problem. Here is a step by step tutorial for that:

http://www.yourtechchick.com/hadoop/how-to-completely-remove-and-uninstall-hdp-components-hadoop-uni...

Hope that helps!