Ambari agents cannot connect to ambari-server after changing keys and certificates..

david_garcia_es · ‎07-21-2017

Hi guys,

After 2 days of Headaches I finally managed to change the certificates and keys of my ambarí-server, and I relaunched it in HTTPS.

Unfortunately, the dashboards doesn't show anything, all the agents, and no heartbeat is received from any service. I restarted all the agents and the server and there is not any progress, so I think its due to some certificciate misunderstanding between server and agents.. communication with server host is ok as you can see in the ping:

[clusteradmin@worker1 ~]$ ping master1
PING master1.pf0g2dnjye1ujcvq5102dppltf.ax.internal.cloudapp.net (172.31.0.4) 56(84) bytes of data.
64 bytes from master1.pf0g2dnjye1ujcvq5102dppltf.ax.internal.cloudapp.net (172.31.0.4): icmp_seq=1 ttl=64 time=0.539 ms

But after diving into agents log I can see this trace being repeated:

INFO 2017-07-21 14:33:46,880 NetUtil.py:60 - Connecting to https://master1:8440/ca
ERROR 2017-07-21 14:33:46,885 NetUtil.py:84 - EOF occurred in violation of protocol (_ssl.c:765)
ERROR 2017-07-21 14:33:46,886 NetUtil.py:85 - SSLError: Failed to connect. Please check openssl library versions.
Refer to: https://bugzilla.redhat.com/show_bug.cgi?id=1022468 for more details.
WARNING 2017-07-21 14:33:46,886 NetUtil.py:112 - Server at https://master1:8440 is not reachable, sleeping for 10 seconds...
INFO 2017-07-21 14:33:56,886 NetUtil.py:60 - Connecting to https://master1:8440/ca
ERROR 2017-07-21 14:33:56,892 NetUtil.py:84 - EOF occurred in violation of protocol (_ssl.c:765)
ERROR 2017-07-21 14:33:56,892 NetUtil.py:85 - SSLError: Failed to connect. Please check openssl library versions.

Taking into account than Openssl version is the latest possible, maybe ¿should I put some keys or certificates on the agents? but what files? my crt or my ca.crt? my public key into their authorized_key files??

I am not very strong on ssh insights, so any help will be apreciatted.

Thanks in advance!!

jsensharma · ‎07-21-2017

@david garcia

You mentioned that You finally managed to change the certificates and keys of my ambarí-server. Is that the ambari UI HTTPS?

The error is between the Ambari Agent and Ambari Server on port 8440. Which usually happens by the ambari generated default certificates. If that certificate is corrupted or expired then you can refer to the following article to know how to fix that: https://community.hortonworks.com/articles/68799/steps-to-fix-ambari-server-agent-expired-certs.html

.

I see a similar post of yours here, Which looks duplicate of this HCC thread, Can you please close one of the thread.

https://community.hortonworks.com/questions/114808/ambari-agents-cannot-reach-ambari-server-after-ch...