Ambari agent fail to start with errors after upgrade from Ambari 2.2.1.1 to 2.4.0.1-1

[#1]

We have upgraded the Ambari Server and agent from 2.2.1.1 to 2.4.0.1-1

Ambari Server and Agent on the same server is running fine. We have additional 2 nodes and we are getting following error on both the nodes while starting Ambari agent.

- we have updated ambari-agent.ini to point to Ambari Server

- We do not see /var/log/ambari-agent/ambari-agent.log file created.

- We have tried uninstall and re-install of the ambari-agent but still getting same following error.

% ambari-agent restart --verbose

Restarting ambari-agent
Verifying Python version compatibility...
Using python  /usr/bin/python
ambari-agent is not running. No PID found at /var/run/ambari-agent/ambari-agent.pid
Verifying Python version compatibility...
Using python  /usr/bin/python
Checking for previously running Ambari Agent...
Starting ambari-agent
Verifying ambari-agent process status...
ERROR: ambari-agent start failed. For more details, see /var/log/ambari-agent/ambari-agent.out:
====================
    from Controller import AGENT_AUTO_RESTART_EXIT_CODE
  File "/usr/lib/python2.6/site-packages/ambari_agent/Controller.py", line 37, in <module>
    import AmbariConfig
  File "/usr/lib/python2.6/site-packages/ambari_agent/AmbariConfig.py", line 26, in <module>
    from NetUtil import NetUtil
  File "/usr/lib/python2.6/site-packages/ambari_agent/NetUtil.py", line 22, in <module>
    from HeartbeatHandlers import HeartbeatStopHandlers
  File "/usr/lib/python2.6/site-packages/ambari_agent/HeartbeatHandlers.py", line 21, in <module>
    from ambari_commons.exceptions import FatalException
ImportError: No module named ambari_commons.exceptions
====================
Agent out at: /var/log/ambari-agent/ambari-agent.out
Agent log at: /var/log/ambari-agent/ambari-agent.log

% yum info ambari-agent 
Loaded plugins: security
Installed Packages
Name        : ambari-agent
Arch        : x86_64
Version     : 2.4.0.1
Release     : 1
Size        : 38 M
Repo        : installed
From repo   : Updates-ambari-2.4.0.1
Summary     : Ambari Agent
URL         : http://www.apache.org
License     : (c) Apache Software Foundation
Description : Maven Recipe: RPM Package.

[#2]

@Raviraj Murdeshwar

please uninstall ambari-agent, remove all associated directories like /etc/ambari-agent, /var/lib/ambari-agent, etc.

Then delete the following directory

/usr/lib/python2.6/site-packages/ambari-agent

after that install ambari-agent again.

[#3]

@Raviraj

Please check /var/log/ambari-agent/ambari-agent.out. and provide me more details.

[#4]

I am working with Raviraj and performed the recommended steps on his behalf, issue still exists, different error message this time though.

As suggested:

Uninstaled ambari-agent, removed following directories and successfully re-installed ambari-agent:

/etc/ambari-agent

/var/lib/ambari-agent,

/var/run/ambari-agent

/usr/lib/amrbari-agent

/usr/lib/python2.6/site-packages/ambari-agent

While starting the agent it now throws different error:

Starting ambari-agent Verifying ambari-agent process status... ERROR: ambari-agent start failed. For more details, see /ambari-agent.out: ==================== Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/ambari_agent/AmbariAgent.py", line 24, in <module> from Controller import AGENT_AUTO_RESTART_EXIT_CODE File "/usr/lib/python2.6/site-packages/ambari_agent/Controller.py", line 41, in <module> from ambari_agent.ActionQueue import ActionQueue File "/usr/lib/python2.6/site-packages/ambari_agent/ActionQueue.py", line 37, in <module> from ambari_commons.str_utils import split_on_chunks ImportError: cannot import name split_on_chunks ====================

The ambari-agent.out file also show same above messages that we see in console

,

Uninstaled ambari-agent, removed following directories and successfully re-installed ambari-agent:

/etc/ambari-agent

/var/lib/ambari-agent,

/var/run/ambari-agent

/usr/lib/amrbari-agent

/usr/lib/python2.6/site-packages/ambari-agent

While starting the agent it now throws different error:

Starting ambari-agent Verifying ambari-agent process status... ERROR: ambari-agent start failed. For more details, see /ambari-agent.out: ==================== Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/ambari_agent/AmbariAgent.py", line 24, in <module> from Controller import AGENT_AUTO_RESTART_EXIT_CODE File "/usr/lib/python2.6/site-packages/ambari_agent/Controller.py", line 41, in <module> from ambari_agent.ActionQueue import ActionQueue File "/usr/lib/python2.6/site-packages/ambari_agent/ActionQueue.py", line 37, in <module> from ambari_commons.str_utils import split_on_chunks ImportError: cannot import name split_on_chunks ====================

The ambari-agent.out file also show same above messages that we see in console

[#5 · Accepted solution]

@Samir Buche

Please follow the link may be it's help you.

1. yum erase ambari-agent
2. rm -rf /var/lib/ambari-agent
3. rm -rf /var/log/ambari-agent
4. rm -rf /usr/lib/python2.6/site-packages/ambari_agent
5. yum install -y ambari-agent

Or please follow the link.

https://community.hortonworks.com/questions/25382/hdp-exporess-upgrade-failing-with-importerror-cann...

[#6]

@ Samir Buche

Issue is resolved or still there?

[#7]

Sadly , I upgrade the agent recently from version 2.5.0.3 to 2.5.1.0 this Issue is still there

[#8]

Thanks Ashnee, It worked after removing all files/configuration from previous installs was causing problem.