03-27-2019 04:30 AM
I am trying to enable SSL in transit for my cloudera cluster using the document https://www.cloudera.com/documentation/enterprise/latest/topics/how_to_configure_cm_tls.html
I am able to successfully configure till the step "Enable Server Certificate Verification on Cloudera Manager Agents" , however once i completed "Configure Agent Certificate Authentication" , i am receiving the below error and all the hosts are in bad health state, could you please help ?
[27/Mar/2019 11:12:37 +0000] 1022 MainThread agent ERROR Heartbeating to cmhost.antuit.internal:7182 failed.
Traceback (most recent call last):
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/agent.py", line 1388, in _send_heartbeat
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 139, in __init__
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/httpslib.py", line 80, in connect
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/SSL/Connection.py", line 305, in connect
ret = self.connect_ssl()
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/SSL/Connection.py", line 292, in connect_ssl
return m2.ssl_connect(self.ssl, self._timeout)
SSLError: sslv3 alert certificate unknown
1. I am using a private intermediate CA to sign the certificates for each host.
2. I have imported both root and intermediate CA certs into jssecacerts in the cloudera manager host
3. I am able to manually verify the signed certs
cat ca.cert.pem intermediate.cert.pem > verify.pem
sudo openssl verify -CAfile verifier.pem cmhost.XX.YY.pem
03-27-2019 06:20 AM
Did you follow the steps as explained in Enable Server Certificate Verification on Cloudera Manager Agents ?
The agent configuration needs to be given a PEM file for the verify_cert_file configuration property which holds the root CA certificate.
To verify the configuration, please run this command on one of the cluster nodes, with <filename> being the full path to file as specified in verify_cert_file:
# openssl s_client -CAfile <filename> cmhost.antuit.internal:7182
03-27-2019 09:27 PM
Yes i have completed the steps in "Enable server certificate verification" . Please find details below,
Note: The verifier.pem file has both RootCA and IntermediateCA certificates and cmhost.antuit.internal.pem has the signed certificate + IntermediateCA certificate
# A file of CA certificates in PEM format. The file can contain several CA
# certificates identified by
# -----BEGIN CERTIFICATE-----
# ... (CA certificate in base64 encoding) ...
# -----END CERTIFICATE-----
# sequences. Before, between, and after the certificates text is allowed which
# can be used e.g. for descriptions of the certificates.
# The file is loaded once, the first time an HTTPS connection is attempted. A
# restart of the agent is required to pick up changes to the file.
# Note that if neither verify_cert_file or verify_cert_dir is set, certificate
# verification will not be performed.
[root@cmhost pki]# openssl s_client -connect cmhost.antuit.internal:7182 -CAfile verifier.pem -cert cmhost.antuit.internal.pem -key agent.key
Enter pass phrase for agent.key:
depth=2 C = IN, ST = KA, L = BNG, O = Antuit, OU = DE, CN = Antuit Root CA
depth=1 C = IN, ST = KA, O = Antuit, OU = DE, CN = Antuit Inter CA
depth=0 C = IN, ST = KA, L = BNG, O = Antuit, OU = DE, CN = cmhost.antuit.internal
140606215886736:error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown:s3_pkt.c:1493:SSL alert number 46
140606215886736:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
i:/C=IN/ST=KA/O=Antuit/OU=DE/CN=Antuit Inter CA
1 s:/C=IN/ST=KA/O=Antuit/OU=DE/CN=Antuit Inter CA
i:/C=IN/ST=KA/L=BNG/O=Antuit/OU=DE/CN=Antuit Root CA
03-28-2019 12:48 AM
Please make the verifier.pem file contain only the root CA certificate. Then list contents with
# openssl x509 -text -in /opt/cloudera/security/pki/verifier.pem
And repeat the connection test with this exact command
# openssl s_client -connect cmhost.antuit.internal:7182 -CAfile /opt/cloudera/security/pki/verifier.pem