abari-agent is failing to connect ambari-server?

[#1]

Hi Community team,

Amabari-agent is failing to connect Aambari-server

In agent logs :

INFO 2017-10-16 05:50:56,334 PingPortListener.py:50 - Ping port listener started on port: 8670 INFO 2017-10-16 05:50:56,407 main.py:287 - Connecting to Ambari server at https://bdlambtst01.isus.emc.com:8440 (10.146.9.185) INFO 2017-10-16 05:50:56,407 main.py:287 - Connecting to Ambari server at https://bdlambtst01.isus.emc.com:8440 (10.146.9.185) INFO 2017-10-16 05:50:56,407 NetUtil.py:59 - Connecting to https://bdlambtst01.isus.emc.com:8440/ca

in /var/log/messages:

Oct 16 05:49:09 bdlhdctst04 python: detected unhandled Python exception in '/usr/lib/python2.6/site-packages/ambari_agent/main.py' Oct 16 05:50:01 bdlhdctst04 systemd: Started Session 4823 of user root. Oct 16 05:50:01 bdlhdctst04 systemd: Starting Session 4823 of user root. Oct 16 05:50:56 bdlhdctst04 python2.7: detected unhandled Python exception in '/usr/lib/python2.6/site-packages/ambari_agent/main.py'

i have verified the network connectivity is good agent --> server & server --> agent?

i have observed that there are no files are present in /var/lib/ambari-agent/data location?

Any help would be appreciate this situation?

Regards,

Subhash.

[#2]

@subhash parise

Your Ambari is SSL enabled did you by chance also enable 2 ways SSL between Ambari Server and Ambari Agents? see doc

Here is a HCC document that can also help you achieve that.

Enable 2 way SSL between Ambari-Server and Ambari-agents in Ambari 2.5

Hope that helps

[#3]

Hi Geo,

all the other nodes are working fine except this node and we didn't change any thing from ambari side.

[#4 · Accepted solution]

@subhash parise

On the offending node, do the following, bizzare thee are no files in /var/lib/ambari-agent/data:

Stop and remove ambari-agent

ambari-agent stop 
yum erase ambari-agent 
rm -rf /var/lib/ambari-agent
rm -rf /var/run/ambari-agent
rm -rf /usr/lib/amrbari-agent
rm -rf /etc/ambari-agent
rm -rf /var/log/ambari-agent
rm -rf /usr/lib/python2.6/site-packages/ambari* 

Re-install the Ambari Agent

yum install ambari-agent
vi /etc/ambari-agent/conf/ambari-agent.ini 

Change hostname to Ambari Server

[server]
hostname={Ambari-server_host_FQDN}
url_port=8440
secured_url_port=8441
connect_retry_delay=10
max_reconnect_retry_delay=30 

Restart the agent ambari-agent start

That should resolve the issue