Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

SSLError: certificate verify failed

Re: SSLError: certificate verify failed

Super Guru

Re: SSLError: certificate verify failed

Explorer

So I've moved the HAproxy and Keepalived to another set of hosts.  Same issue with certs however I did try to replace the cm-r01nn01 and cm-r01nn02 (Name Node's) with the SAN SSL Certificate that I've generated.  Here is the writeup:

 

==> /var/log/cloudera-scm-agent/cloudera-scm-agent.log <==
self.cfg.max_cert_depth)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 139, in __init__
self.conn.connect()
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/httpslib.py", line 69, in connect
sock.connect((self.host, self.port))
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/SSL/Connection.py", line 309, in connect
ret = self.connect_ssl()
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/M2Crypto/SSL/Connection.py", line 295, in connect_ssl
return m2.ssl_connect(self.ssl, self._timeout)
SSLError: certificate verify failed

 

Replace the certificate cm-r01nn01 and cm-r01nn02 with the SAN SSL Cert one as follows:

 

 

[root@cm-r01nn01 yum.repos.d]# keytool -list -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit | grep -Ei "cm|srv"
cm-r01wn07.mws.mds.xyz, Jul 13, 2019, PrivateKeyEntry,
cm-r01wn04.mws.mds.xyz, Jul 13, 2019, PrivateKeyEntry,
cm-r01wn01.mws.mds.xyz, Jul 13, 2019, PrivateKeyEntry,
cm-r01wn08.mws.mds.xyz, Jul 13, 2019, PrivateKeyEntry,
cm-r01nn01.mws.mds.xyz, Jul 12, 2019, PrivateKeyEntry,
cm-r01wn05.mws.mds.xyz, Jul 13, 2019, PrivateKeyEntry,
cm-r01wn02.mws.mds.xyz, Jul 13, 2019, PrivateKeyEntry,
cm-r01en01.mws.mds.xyz, Jul 12, 2019, PrivateKeyEntry,
acraizfnmt-rcm, Mar 26, 2019, trustedCertEntry,
cm-r01nn02.mws.mds.xyz, Jul 12, 2019, PrivateKeyEntry,
cm-r01wn06.mws.mds.xyz, Jul 13, 2019, PrivateKeyEntry,
cm-r01wn03.mws.mds.xyz, Jul 13, 2019, PrivateKeyEntry,
srv-c01.mws.mds.xyz, Jul 18, 2019, PrivateKeyEntry,
cm-r01en02.mws.mds.xyz, Jul 12, 2019, PrivateKeyEntry,
cm-c01.mws.mds.xyz, Jul 12, 2019, PrivateKeyEntry,

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
[root@cm-r01nn01 yum.repos.d]#
[root@cm-r01nn01 yum.repos.d]#
[root@cm-r01nn01 yum.repos.d]# keytool -list -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit | grep -Ei "cm|srv"|grep -Ei r01nn01

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
cm-r01nn01.mws.mds.xyz, Jul 12, 2019, PrivateKeyEntry,
[root@cm-r01nn01 yum.repos.d]# keytool -delete -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit -alias cm-r01nn01.mws.mds.xyz ^C
[root@cm-r01nn01 yum.repos.d]#
[root@cm-r01nn01 yum.repos.d]#
[root@cm-r01nn01 yum.repos.d]#
[root@cm-r01nn01 yum.repos.d]# cd /root/srv-c01/
[root@cm-r01nn01 srv-c01]# ls -altri
total 40
201326721 dr-xr-x---. 11 root root 4096 Jul 17 22:04 ..
134330464 drwxr-xr-x. 2 root root 4096 Jul 18 22:39 1
2230259 -rw-r--r--. 1 root root 2422 Jul 18 22:46 srv-c01.mws.mds.xyz.keystore.jks
2230300 -rw-r--r--. 1 root root 2801 Jul 18 22:49 srv-c01.mws.mds.xyz.keystore.p12
2230651 -rw-r--r--. 1 root root 1763 Jul 18 22:49 srv-c01.mws.mds.xyz.cert.pem
2230652 -rw-r--r--. 1 root root 1860 Jul 18 22:49 srv-c01.mws.mds.xyz.key.pem
2230653 -rw-r--r--. 1 root root 1505 Jul 18 22:51 srv-c01.mws.mds.xyz.pem
2230654 -rw-r--r--. 1 root root 1679 Jul 18 23:00 srv-c01.mws.mds.xyz.key.nopass.pem
2230257 drwxr-xr-x. 3 root root 4096 Jul 18 23:00 .
2230655 -rw-r--r--. 1 root root 3442 Jul 18 23:00 srv-c01.mws.mds.xyz-haproxy.pem
[root@cm-r01nn01 srv-c01]# history|grep srv-c01.mws.mds.xyz.pem|grep openssl|tail
876 openssl x509 -in ./srv-c01.mws.mds.xyz.pem -noout -text
1029 history|grep srv-c01.mws.mds.xyz.pem|grep openssl|tail
[root@cm-r01nn01 srv-c01]# openssl x509 -in ./srv-c01.mws.mds.xyz.pem -noout -text
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 1594172762 (0x5f05255a)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US, ST=California, L=Los Angeles, O=MDS, OU=MDS, CN=srv-c01.mws.mds.xyz
Validity
Not Before: Jul 19 02:46:18 2019 GMT
Not After : Jul 16 02:46:18 2029 GMT
Subject: C=US, ST=California, L=Los Angeles, O=MDS, OU=MDS, CN=srv-c01.mws.mds.xyz
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:c5:a9:00:83:12:9e:02:86:32:4e:2b:a7:c6:1a:
6b:9d:e3:56:00:53:22:01:d8:db:83:cd:14:79:6a:
85:27:20:f6:5d:86:0e:0b:af:df:46:dd:c3:23:72:
f0:bf:38:3e:cd:9f:92:e6:65:81:7b:26:32:50:fc:
81:0e:7b:dd:b4:61:6f:a7:56:ec:c8:fe:89:72:ec:
e5:e0:63:61:92:77:0b:36:41:98:93:14:6d:53:a0:
24:fb:fb:77:40:98:5b:2f:d2:3c:65:4f:8b:65:33:
e5:db:14:ce:01:d2:4f:9f:e4:c6:c8:35:50:09:a2:
f3:48:0a:ac:06:fd:66:42:30:10:a4:e7:fa:a8:2b:
0b:2b:ef:ce:83:82:4e:0d:86:34:ce:0c:8d:0c:a2:
f5:88:4d:38:9f:3b:dd:2e:6e:e3:8c:60:69:da:8d:
a4:d4:db:d5:cd:26:91:95:ca:a2:47:de:3c:f3:8f:
52:b8:e5:b0:09:26:af:77:fb:a3:5b:40:f6:e8:1b:
66:d7:b7:1b:da:2c:6c:34:99:76:de:c4:9b:80:69:
25:d5:12:2f:cb:9b:c5:d2:7e:15:a7:50:5f:54:5c:
9d:6b:8c:c0:9c:03:3f:96:f3:8a:2c:a6:05:ec:a4:
d3:83:84:61:13:da:57:6d:e8:8c:93:d9:40:38:24:
96:c9
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication, Code Signing, E-mail Protection, Time Stamping, OCSP Signing
X509v3 Subject Alternative Name:
DNS:srv-c01.mws.mds.xyz, DNS:cm-r01nn01.mws.mds.xyz, DNS:cm-r01nn02.mws.mds.xyz
X509v3 Subject Key Identifier:
F6:EA:97:6F:82:20:84:75:E9:63:71:2F:16:D6:41:8B:64:05:07:0D
Signature Algorithm: sha256WithRSAEncryption
4f:35:6d:18:dc:5c:4a:65:db:8c:62:75:0b:f8:da:2b:14:72:
22:f7:3a:ba:15:17:58:41:46:3b:6b:6e:40:db:6b:be:e5:07:
82:d1:37:0a:d6:4e:96:14:f6:87:ca:ff:d3:5f:a9:94:de:81:
e7:a1:28:94:0a:19:0b:f4:dc:ed:0a:a5:77:78:20:53:3f:3f:
03:54:67:a0:c4:a1:de:49:7d:e8:fc:2d:76:bd:7b:a5:98:cd:
45:7e:ba:21:79:e2:91:7d:f3:e9:d6:5d:b7:91:34:30:3a:e4:
3a:38:e9:33:9b:26:2e:3e:6c:c9:3d:5d:48:81:cb:35:2f:ff:
7a:ff:22:c2:f8:b5:a2:01:d0:54:7f:f2:08:33:89:78:80:af:
72:2d:d7:df:61:f0:4a:7f:d2:19:0d:c6:0c:51:ee:4e:c1:ed:
8d:8b:4f:82:17:47:6b:03:1a:f2:8b:00:cc:17:8a:75:ca:72:
c0:a4:a7:12:87:32:16:89:15:2c:80:d1:07:fd:37:e8:bf:f5:
87:6b:a2:dd:9d:a4:c4:2c:68:f8:d9:15:dd:3c:40:6d:8b:e0:
6d:c4:87:6d:39:a9:6b:91:f6:0a:bc:7c:63:e7:f0:37:cb:7a:
5f:35:6c:5c:f9:bb:cb:58:1a:b9:9c:49:ab:24:ac:2a:c9:2d:
3f:b2:2f:68
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]# keytool -list -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit | grep -Ei "cm|srv"|grep -Ei r01nn01

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
cm-r01nn01.mws.mds.xyz, Jul 12, 2019, PrivateKeyEntry,
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]# keytool -list -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit | grep -Ei "cm|srv"|grep -Ei r01nn01

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
cm-r01nn01.mws.mds.xyz, Jul 12, 2019, PrivateKeyEntry,
[root@cm-r01nn01 srv-c01]# keytool -delete -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit -alias cm-r01nn01.mws.mds.xyz

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
[root@cm-r01nn01 srv-c01]# keytool -delete -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit -alias cm-r01nn02.mws.mds.xyz

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]# keytool -importkeystore -srckeystore srv-c01.mws.mds.xyz.keystore.jks -destkeystore srv-c01.mws.mds.xyz.keystore.p12 -deststoretype PKCS12 -destalias cm-r01nn01.mws.mds.xyz -srcalias srv-c01.mws.mds.xyz -deststorepass srv-c01.mws.mds.xyz -destkeypass srv-c01.mws.mds.xyz
Importing keystore srv-c01.mws.mds.xyz.keystore.jks to srv-c01.mws.mds.xyz.keystore.p12...
Enter source keystore password:
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]# keytool -list -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit | grep -Ei "cm|srv"|grep -Ei r01nn

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
[root@cm-r01nn01 srv-c01]# keytool -delete -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit -alias cm-r01nn01.mws.mds.xyz
keytool error: java.lang.Exception: Alias <cm-r01nn01.mws.mds.xyz> does not exist
[root@cm-r01nn01 srv-c01]# keytool -list -keystore srv-c01.mws.mds.xyz.keystore.p12
Enter keystore password:
Keystore type: PKCS12
Keystore provider: SUN

Your keystore contains 2 entries

srv-c01.mws.mds.xyz, Jul 18, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): 01:2D:6C:36:E6:7D:76:2B:45:66:91:7F:E8:B9:C2:61:A4:22:FB:D6
cm-r01nn01.mws.mds.xyz, Jul 26, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): 01:2D:6C:36:E6:7D:76:2B:45:66:91:7F:E8:B9:C2:61:A4:22:FB:D6
[root@cm-r01nn01 srv-c01]# keytool -delete -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit -alias cm-r01nn01.mws.mds.xyz
keytool error: java.lang.Exception: Alias <cm-r01nn01.mws.mds.xyz> does not exist
[root@cm-r01nn01 srv-c01]# keytool -delete -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit -destalias cm-r01nn01.mws.mds.xyz
Enter alias name: cm-r01nn01.mws.mds.xyz
keytool error: java.lang.Exception: Alias <cm-r01nn01.mws.mds.xyz> does not exist
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]# keytool -delete -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass cm-r01nn01.mws.mds.xyz -alias cm-r01nn01.mws.mds.xyz
keytool error: java.io.IOException: Keystore was tampered with, or password was incorrect
[root@cm-r01nn01 srv-c01]# keytool -delete -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass srv-c01.mws.mds.xyz -alias cm-r01nn01.mws.mds.xyz
keytool error: java.io.IOException: Keystore was tampered with, or password was incorrect
[root@cm-r01nn01 srv-c01]# keytool -delete -keystore srv-c01.mws.mds.xyz.keystore.p12 -storepass srv-c01.mws.mds.xyz -alias cm-r01nn01.mws.mds.xyz [root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]# keytool -list -keystore srv-c01.mws.mds.xyz.keystore.p12 Enter keystore password:
keytool error: java.io.IOException: keystore password was incorrect
[root@cm-r01nn01 srv-c01]# keytool -list -keystore srv-c01.mws.mds.xyz.keystore.p12
Enter keystore password:
Keystore type: PKCS12
Keystore provider: SUN

Your keystore contains 1 entry

srv-c01.mws.mds.xyz, Jul 18, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): 01:2D:6C:36:E6:7D:76:2B:45:66:91:7F:E8:B9:C2:61:A4:22:FB:D6
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]# history|grep import|grep srv-c01.mws.mds.xyz.keystore.p12
699 keytool -importkeystore -srckeystore srv-c01.mws.mds.xyz.keystore.jks -destkeystore srv-c01.mws.mds.xyz.keystore.p12 -deststoretype PKCS12 -srcalias srv-c01.mws.mds.xyz -deststorepass srv-c01.mws.mds.xyz
869 keytool -importkeystore -srckeystore srv-c01.mws.mds.xyz.keystore.jks -destkeystore srv-c01.mws.mds.xyz.keystore.p12 -deststoretype PKCS12 -srcalias srv-c01.mws.mds.xyz -deststorepass srv-c01.mws.mds.xyz -destkeypass srv-c01.mws.mds.xyz
1034 keytool -importkeystore -srckeystore srv-c01.mws.mds.xyz.keystore.jks -destkeystore srv-c01.mws.mds.xyz.keystore.p12 -deststoretype PKCS12 -destalias cm-r01nn01.mws.mds.xyz -srcalias srv-c01.mws.mds.xyz -deststorepass srv-c01.mws.mds.xyz -destkeypass srv-c01.mws.mds.xyz
1044 history|grep import|grep srv-c01.mws.mds.xyz.keystore.p12
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]# keytool -importkeystore -srckeystore srv-c01.mws.mds.xyz.keystore.jks -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -srcalias srv-c01.mws.mds.xyz -deststorepass changeit -srcstorepass srv-c01.mws.mds.xyz -destalias cm-r01nn01.mws.mds.xyz
Importing keystore srv-c01.mws.mds.xyz.keystore.jks to /etc/pki/ca-trust/extracted/java/jssecacerts...

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]# keytool -importkeystore -srckeystore srv-c01.mws.mds.xyz.keystore.jks -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -srcalias srv-c01.mws.mds.xyz -deststorepass changeit -srcstorepass srv-c01.mws.mds.xyz -destalias cm-r01nn02.mws.mds.xyz
Importing keystore srv-c01.mws.mds.xyz.keystore.jks to /etc/pki/ca-trust/extracted/java/jssecacerts...

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#
[root@cm-r01nn01 srv-c01]#





[root@cm-r01nn01 srv-c01]# keytool -list -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit -alias srv-c01.mws.mds.xyz -alias cm-r01nn01.mws.mds.xyz -v
Alias name: cm-r01nn01.mws.mds.xyz
Creation date: Jul 26, 2019
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=srv-c01.mws.mds.xyz, OU=MDS, O=MDS, L=Los Angeles, ST=California, C=US
Issuer: CN=srv-c01.mws.mds.xyz, OU=MDS, O=MDS, L=Los Angeles, ST=California, C=US
Serial number: 5f05255a
Valid from: Thu Jul 18 22:46:18 EDT 2019 until: Sun Jul 15 22:46:18 EDT 2029
Certificate fingerprints:
MD5: A7:C0:9E:E4:CC:DC:7E:4B:3A:96:CF:11:58:6C:86:D2
SHA1: 01:2D:6C:36:E6:7D:76:2B:45:66:91:7F:E8:B9:C2:61:A4:22:FB:D6
SHA256: 62:11:52:4E:7C:1E:03:11:20:CA:41:7E:5D:22:64:F8:CE:CC:85:C6:07:06:A9:21:FE:25:F3:71:DD:20:00:49
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions:

#1: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
serverAuth
clientAuth
codeSigning
emailProtection
timeStamping
OCSPSigning
]

#2: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
DNSName: srv-c01.mws.mds.xyz
DNSName: cm-r01nn01.mws.mds.xyz
DNSName: cm-r01nn02.mws.mds.xyz
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: F6 EA 97 6F 82 20 84 75 E9 63 71 2F 16 D6 41 8B ...o. .u.cq/..A.
0010: 64 05 07 0D d...
]
]


Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
[root@cm-r01nn01 srv-c01]#







[root@cm-r01wn08 srv-c01]#
[root@cm-r01wn08 srv-c01]# keytool -importkeystore -srckeystore srv-c01.mws.mds.xyz.keystore.jks -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -srcalias srv-c01.mws.mds.xyz -deststorepass changeit -srcstorepass srv-c01.mws.mds.xyz -destalias cm-r01nn01.mws.mds.xyz
Importing keystore srv-c01.mws.mds.xyz.keystore.jks to /etc/pki/ca-trust/extracted/java/jssecacerts...

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
[root@cm-r01wn08 srv-c01]# keytool -importkeystore -srckeystore srv-c01.mws.mds.xyz.keystore.jks -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -srcalias srv-c01.mws.mds.xyz -deststorepass changeit -srcstorepass srv-c01.mws.mds.xyz -destalias cm-r01nn02.mws.mds.xyz
Importing keystore srv-c01.mws.mds.xyz.keystore.jks to /etc/pki/ca-trust/extracted/java/jssecacerts...

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
[root@cm-r01wn08 srv-c01]#
[root@cm-r01wn08 srv-c01]#
[root@cm-r01wn08 srv-c01]#
[root@cm-r01wn08 srv-c01]# keytool -list -keystore /etc/pki/ca-trust/extracted/java/jssecacerts -storepass changeit | grep -Ei "cm|srv" srv-c01.mws.mds.xyz, Jul 21, 2019, PrivateKeyEntry,
acraizfnmt-rcm, Mar 26, 2019, trustedCertEntry,
cm-r01wn08.mws.mds.xyz, Jul 10, 2019, trustedCertEntry,
cm-r01nn02.mws.mds.xyz, Jul 26, 2019, PrivateKeyEntry,
cm-r01nn01.mws.mds.xyz, Jul 26, 2019, PrivateKeyEntry,

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/pki/ca-trust/extracted/java/jssecacerts -destkeystore /etc/pki/ca-trust/extracted/java/jssecacerts -deststoretype pkcs12".
[root@cm-r01wn08 srv-c01]#
[root@cm-r01wn08 srv-c01]#

But no luck.  Again, is there any way to increase the debug level on the cloudera-scm-agent so it will print further information detailing what private and public key files it's opening up?

 

Thx,
TK

Highlighted

Re: SSLError: certificate verify failed

Super Guru

@TCloud,

The exception is in the agent and indicates to us that the agent is not able to verify the certificate that was returned by Cloudera Manager during the TLS handshake.

In order to know why, we should look at what host the agent tried to contact (server_host in config.ini) and what certificates were listed in the SAN of the server certificate.

You can use the following command to see what certificate is returned:

openssl s_client -connect $(grep "server_host" /etc/cloudera-scm-agent/config.ini | sed s/server_host=//):7182 </dev/null | openssl x509 -text -noout

Then, check to make sure agent's truststore has the proper certificate that trusts the CM cert.  To test, you can use:

openssl s_client -connect $(grep -v '^#' /etc/cloudera-scm-agent/config.ini | grep "server_host=" | sed s/server_host=//):7182 -CAfile $(grep -v '^#' /etc/cloudera-scm-agent/config.ini | grep "verify_cert_file=" |sed s/verify_cert_file=//) -verify_hostname $(grep -v '^#' /etc/cloudera-scm-agent/config.ini | grep "server_host=" | sed s/server_host=//)</dev/null

 

The above is probably not that elegant, but you should be able to run it as it is.  It will grab your hostname and trust store file from the host's config.ini and then connect to your CM host to do a TLS handshake.  "-verify_hostname" will tell openssl to also do hostname validation to mimic what the agent does.

 

The result code of the above command should give us a better idea of why the handshake is failing.