Support Questions
Find answers, ask questions, and share your expertise

ambari agents cannot reach ambari-server after changing server's ssl stuff

Hi guys,

After 2 days of Headaches I finally managed to change the certificates and keys of my ambarí-server, and I relaunched it in HTTPS.

Unfortunately, the dashboards doesn't show anything, all the agents are like empty and no heartbeat is received from any service. I restarted all the agents and the server and there is not any progress, so I think its due to some certificciate misunderstanding between server and agents..

communication with server host since agents machines is ok as you can see in the ping:

[clusteradmin@worker1 ~]$ ping master1
PING master1.pf0g2dnjye1ujcvq5102dppltf.ax.internal.cloudapp.net (172.31.0.4) 56(84) bytes of data.
64 bytes from master1.pf0g2dnjye1ujcvq5102dppltf.ax.internal.cloudapp.net (172.31.0.4): icmp_seq=1 ttl=64 time=0.539 ms
But after diving into agents log I can see this trace being repeated:
INFO 2017-07-21 14:33:46,880 NetUtil.py:60 - Connecting to https://master1:8440/ca
ERROR 2017-07-21 14:33:46,885 NetUtil.py:84 - EOF occurred in violation of protocol (_ssl.c:765)
ERROR 2017-07-21 14:33:46,886 NetUtil.py:85 - SSLError: Failed to connect. Please check openssl library versions.
Refer to: https://bugzilla.redhat.com/show_bug.cgi?id=1022468 for more details.
WARNING 2017-07-21 14:33:46,886 NetUtil.py:112 - Server at https://master1:8440 is not reachable, sleeping for 10 seconds...
INFO 2017-07-21 14:33:56,886 NetUtil.py:60 - Connecting to https://master1:8440/ca
ERROR 2017-07-21 14:33:56,892 NetUtil.py:84 - EOF occurred in violation of protocol (_ssl.c:765)
ERROR 2017-07-21 14:33:56,892 NetUtil.py:85 - SSLError: Failed to connect. Please check openssl library versions.

Taking into account than Openssl version is the latest possible, maybe ¿should I put some keys or certificates on the agents? but what files? my crt or my ca.crt? my public key into their authorized_key files??

I am not very strong on ssh insights, so any help will be apreciatted.

Thanks in advance!!

5 REPLIES 5

Re: ambari agents cannot reach ambari-server after changing server's ssl stuff

Super Mentor

@david garcia

As you are getting the following error ( which is basically a Security protocol error)

ERROR 2017-07-21 14:33:56,892 NetUtil.py:84 - EOF occurred in violation of protocol (_ssl.c:765)ERROR 2017-07-21 14:33:56,892 NetUtil.py:85 - SSLError: Failed to connect. Please check openssl library versions.
- On Ambari Server "/etc/amabri-agent/conf/ambari.properties" file add the following protocols: security.server.disabled.protocols=SSL|SSLv2|SSLv2Hello|SSLv3|TLSv1

- So can you please try adding the following option to security section in "/etc/amabri-agent/conf/ambari-agent.ini" in all the hosts in the cluster

[security] 
force_https_protocol=PROTOCOL_TLSv1_2

.

Also please let us know :

1. What is your openssl version?

2. Your Ambari version? For some older version of ambari the similar issue is reported here: https://issues.apache.org/jira/browse/AMBARI-17666

3. python version? And please check the protocols supported by your Python version ? Simply by creating following kind of test python script and run the following Python script in your ambari agent machine

"/tmp/testPythonProtocols.py"

#!/usr/bin/env python
import ssl;
for i in dir(ssl): 
  if i.startswith("PROTOCOL"):
    print(i)

. And then run the above file as and please share the output of the follpwing:

# python  /tmp/testPythonProtocols.py

.

Re: ambari agents cannot reach ambari-server after changing server's ssl stuff

New Contributor

That saved me a lot of time, thanks!

Re: ambari agents cannot reach ambari-server after changing server's ssl stuff

Super Mentor

@david garcia

If this resolved/answers your query/issue then please mark this HCC thread as answered by clicking on "Accept" link on the correct answer, That way it will help other HCC users to quickly find the answers.

Re: ambari agents cannot reach ambari-server after changing server's ssl stuff

Mentor

@david garcia

Check out Vipin Rathor answer hope that helps

Re: ambari agents cannot reach ambari-server after changing server's ssl stuff

hi @Jay SenSharma This is my output of the script: PROTOCOL_SSLv2 PROTOCOL_SSLv23 PROTOCOL_SSLv3 PROTOCOL_TLSv1 PROTOCOL_TLSv1_1 PROTOCOL_TLSv1_2 does it give you any clue?