Unable to connect with beeline after enabling HiveServer2 SSL (no name matching error)

[#1]

I am unable to connect to HiveServer2 from beeline after enabling HiveServer2 ssl. In the output below I have changed my FQDN to <hostname> and all passwords to xxxx

beeline> !connect jdbc:hive2://<hostname>:10000/;ssl=true;sslTrustStore=/etc/hive/truststore.jks;trustStorePassword=xxxx
Connecting to jdbc:hive2://<hostname>:10000/;ssl=true;sslTrustStore=/etc/hive/truststore.jks;trustStorePassword=xxxx
Enter username for jdbc:hive2://<hostname>:10000/;ssl=true;sslTrustStore=/etc/hive/truststore.jks;trustStorePassword=xxxx:
Enter password for jdbc:hive2://<hostname>:10000/;ssl=true;sslTrustStore=/etc/hive/truststore.jks;trustStorePassword=xxxx:
18/04/12 09:57:15 [main]: WARN jdbc.HiveConnection: Failed to connect to <hostname>:10000
Error: Could not open client transport with JDBC Uri: jdbc:hive2://<hostname>:10000/;ssl=true;sslTrustStore=/etc/hive/truststore.jks;trustStorePassword=xxxx: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No name matching <hostname> found (state=08S01,code=0)

I enabled HiveServer2 SSL in the Ambari UI, and can see the configuration has propagated here:

/etc/hive/2.6.4.0-91/0/conf.server/hive-site.xml: <name>hive.server2.keystore.password</name>
/etc/hive/2.6.4.0-91/0/conf.server/hive-site.xml- <value>xxxx</value>
--
/etc/hive/2.6.4.0-91/0/conf.server/hive-site.xml: <name>hive.server2.keystore.path</name>
/etc/hive/2.6.4.0-91/0/conf.server/hive-site.xml: <value>/etc/hive/keystore.jks</value>
/etc/hive/2.6.4.0-91/0/conf.server/hive-site.xml- </property>
--
/etc/hive/2.6.4.0-91/0/conf.server/hive-site.xml: <name>hive.server2.use.SSL</name>
/etc/hive/2.6.4.0-91/0/conf.server/hive-site.xml- <value>true</value>
--
/etc/hive/2.6.4.0-91/0/hive-site.xml: <name>hive.server2.keystore.password</name>
/etc/hive/2.6.4.0-91/0/hive-site.xml- <value>xxxx</value>
--
/etc/hive/2.6.4.0-91/0/hive-site.xml: <name>hive.server2.keystore.path</name>
/etc/hive/2.6.4.0-91/0/hive-site.xml: <value>/etc/hive/keystore.jks</value>
/etc/hive/2.6.4.0-91/0/hive-site.xml- </property>
--
/etc/hive/2.6.4.0-91/0/hive-site.xml: <name>hive.server2.use.SSL</name>
/etc/hive/2.6.4.0-91/0/hive-site.xml- <value>true</value>

The keytool commands I ran are:

keytool -genkey -alias hive -keyalg RSA -keystore keystore.jks -keysize 2048
keytool -export -alias hive -file hivecert.crt -keystore keystore.jks
keytool -import -trustcacerts -alias hive -file hivecert.crt -keystore truststore.jks

After this didn't work I tried importing the certificate to the java cacert:
keytool -import -file hivecert.crt -alias hive -keystore /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.ppc64le/jre/lib/security/cacerts -storepass changeit

Here is the top of the 'keytool list' output for the truststore, to show the owner, issuer, and expiry:

Your keystore contains 1 entry

Alias name: hive
Creation date: Apr 11, 2018
Entry type: trustedCertEntry

Owner: CN=<host name>, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Issuer: CN=<host name>, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Serial number: 6ae371ae
Valid from: Wed Apr 11 14:07:43 PDT 2018 until: Tue Jul 10 14:07:43 PDT 2018

Again, <hostname> replaces my FQDN, which is the same value I used in the beeline connect command.

I have restarted Hive and the ambari-server. Any idea what steps I am missing or doing wrong?

[#2]

Hi Eric,

looks you have not signed the cert yet, just use the private key and public key - can you please sign the certificates with following command or use the tinycert.org utility to generate your server(keystore)/client(truststore) certs.

# Step 1 - Generate Key
keytool -keystore server.keystore.jks -alias localhost -validity 365 -keyalg RSA -genkey
# Step 2 - Create  CA & upload the same into Trust Store.
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365
keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert
# Step 3 - CA to singn the Certificate
keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 365 -CAcreateserial -passin pass:test1234
keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore server.keystore.jks -alias localhost -import -file cert-signed
%above commands are example extracted from Kafka documentation to generate self signed certificates - is same for any self signed certificate generationthen perform the same thing.

Hope this helps !!

[#3]

@bkosaraju

Thanks for the suggestion, though I'm still unable to get it to work. There are a few steps that I'm not clear on, and which I may not be doing correctly.


1. When I create the keystore I am using the name of the host that HiveServer2 is on as the CN, and leaving all other fields as the default 'Unknown'. Is this correct?

CN=<hostname>, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown

2. When I run "openssl req -new -x509 -keyout ca-key -out ca-cert -days 365", I am leaving all the fields as the default except for the common name, which I'm setting to the same host name I used when creating the keystore.

Common Name (eg, your name or your server's hostname) []: <hostname>


3. Do I need to import anything into the java cacert? It doesn't seem to make a difference if I do.

4. When connecting through beeline, should I specify the host name, the host IP, or localhost?

!connect jdbc:hive2://<hostname>:10000/;ssl=true;sslTrustStore=/etc/hive/client.truststore.jks;trustStorePassword=xxxxx

!connect jdbc:hive2://<host IP>:10000/;ssl=true;sslTrustStore=/etc/hive/client.truststore.jks;trustStorePassword=xxxxx

!connect jdbc:hive2://localhost:10000/;ssl=true;sslTrustStore=/etc/hive/client.truststore.jks;trustStorePassword=xxxxx

When I use the host IP I get this error,

javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names present (state=08S01,code=0)

When I use "localhost" I get this error,

javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No name matching localhost found (state=08S01,code=0)

I get the same "No name matching" error when I use the host name.
5. When prompted by beeline for a user name and password, what am I expected to put? Leaving it empty and putting the host name do not modify the error I hit.

[#4]

Hi Eric,

In response to "No Subject Alternative name", as part of cert validation this becomes a vital element, and that can be generated using the extention whle you create certificate " -ext san=dns:<your server Nama> " or " -ext san=ip:<your Server Ip> or both -ext san=dns:server.com,ip:11.11.11.11"

this will be used on validating the certificate, against the host so that certificate legitimacy will be tested.

keytool -keystore server.keystore.jks -alias localhost -validity 365-keyalg RSA -genkey -ext san=dns:<server.com>,ip:<11.11.11.11>

please ensure you repeat all remaining steps that sign a cert and import into keystore and truststore as the key changed.

this will clear the error you are facing now.

in response to your other questions :

1/2. You may leave other fields as unknown, but those are to define where the certificate really belongs to.

3. some application process will directly relay on the cacert (keystore) as default store and may not be possible in application to pass the certstore externally, so it is good practice to have the certs added into the default java cacerts.

for instance if you dont want to use the parameteres trust store location and truststore password that will be fall back to cacerts to look up the certificate in the jdbc URL (I suppose but not tested this senario)

4. Please use the same host name what you have provided in certificate CN, as that will performing the validation against that cert.

Hope this helps !!

[#5]

I finally got back to this and with a fresh set of eyes (because I'd forgotten much of what I had done previous) discovered that zookeeper dynamic service discovery was likely what was causing me grief. Once I disabled it I am able to establish the connection using beeline.

I did search online to see if there was a way to keep the zookeeper feature active AND use HiveServer2 SSL, but couldn't find evidence that anybody has attempted it. It's not critical at the moment, but if anybody knows what it might take to get that to work, please pass the info along. I don't think I need to enable zookeeper SSL.

The JDBC connection string when zookeeper dynamic service discovery is enabled looks something like this:

jdbc:hive2://my.pretend.host.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

Should it just be a matter of appending the same ssl options that I use when connecting without dynamic service discovery enabled?
;ssl=true;sslTrustStore=/path/to/client.truststore.pfx;trustStorePassword=temp4now