beeline connect via Knox - SSL issue

[#1]

Hi,

I am trying to connect to Hive through Knox, via beeline (HDP2.2.4, Knox0.4)

Based on http://hortonworks.com/hadoop-tutorial/secure-jdbc-odbc-clients-access-hiveserver2-using-apache-knox... I set the described config parameters accordingly, but the chapter with the SSL certification I don't know what is meant there.

I have to use a self-signed certificate, therefore I just tried exactly the same sslTrustStore and sslTrustStorePassword values as in the document, but it is failing with:

16/02/14 15:40:11 [main]: WARN jdbc.Utils: ***** JDBC param deprecation *****
16/02/14 15:40:11 [main]: WARN jdbc.Utils: The use of hive.server2.transport.mode is deprecated.
16/02/14 15:40:11 [main]: WARN jdbc.Utils: Please use transportMode like so: jdbc:hive2://<host>:<port>/dbName;transportMode=<transport_mode_value>
16/02/14 15:40:11 [main]: WARN jdbc.Utils: ***** JDBC param deprecation *****
16/02/14 15:40:11 [main]: WARN jdbc.Utils: The use of hive.server2.thrift.http.path is deprecated.
16/02/14 15:40:11 [main]: WARN jdbc.Utils: Please use httpPath like so: jdbc:hive2://<host>:<port>/dbName;httpPath=<http_path_value>
Error: Could not create an https connection to jdbc:hive2://<knox-host>:8443/;ssl=true;sslTrustStore=/var/lib/knox/data/security/keystores/gateway.jks;trustStorePassword=knox?hive.server2.transport.mode=http;hive.server2.thrift.http.path=gateway/default/hive. Keystore was tampered with, or password was incorrect (state=08S01,code=0)

My connect string:

beeline> !connect jdbc:hive2://<knox-host>:8443/;ssl=true;sslTrustStore=/var/lib/knox/data/security/keystores/gateway.jks;trustStorePassword=knox?hive.server2.transport.mode=http;hive.server2.thrift.http.path=gateway/default/hive

The referenced documentation says in Step 4:

In the example here, I am connecting to Knox on HDP 2.1 Sandbox which uses a self-signed certificate for SSL. I have exported this certificate to a file in /root/truststore.jks and set a password to this file

But what exactly means "this certificate" and what is its password to export it into another file, some default values there ?!?!

What am I missing to create a beeline-via-Knox connection successfully ?!?!

[#2 · Accepted solution]

Hi,

after some searching (and thanks to this post), the SSL truststore access problem is solved.

Just replace the value for "trustStorePassword" by your knox-master-secret set during installation of Knox.

[#3]

I know that this question already has an answer and I do not mean to troll or demean anyone's answer. I came across this post while searching for information about this very same thing and came up with a similar solution, but one that does not distribute the Knox server trustStore or the master secret key. To achieve the same thing, do the following

1. Export a server certificate from the Knox self-signed cert that you will distribute to users/clients. On the Knox server:

# cd /usr/hdp/current/knox-server/data/security/keystores
# keytool -exportcert -file knox.crt -keystore ./gateway.jks -storepass <master-secret-password>

2. On the client machines (from which you will be connecting to hive through beeline) import the Knox cert into a user specific trustStore. If the .jks file into which you are importing this cert already exists you will need to enter the password that you used when you created it. If the jks file into which are importing does not yet exist it will ask you for a new password. DO NOT LOSE THIS PASSWORD you will need it when including the trustStore in the beeline connection string.

$ keytool -import -keystore myLocalTrustStore.jks -file knox.crt

Now, you can connect to beeline as follows and it will prompt you for the username and password for the authentication implementation that you used when configuring Knox.

$ beeline -u 'jdbc:hive2://knox-server-hostname:8443/database-name/;ssl=true;sslTrustStore=/path/to/myLocalTrustStore.jks;trustStorePassword=<your-trust-store-passwd>;transportMode=http;httpPath=gateway/default/hive'

[#4]

Step 1 requires the following syntax tweak to include the alias otherwise you get an error like:

keytool error: java.lang.Exception: Alias <mykey> does not exist

keytool -exportcert -file knox.crt -keystore ./gateway.jks -storepass <master-secret-password> -alias <alias-used-to-create-key>

[#5]

@ Ryan Chaplin/@Mark Petronic

I am unable to connect on hive through knox using zokeeper dynamic discovery. Below details of my cluster.

HDP: 2.6

Kerberos Enabled: YES ( Windows AD 2012 R2)

Authentication: AD with LDAPS ( Windows AD 2012 R2)

Ranger Enabled: YES

RANGER Usersync and GroupSync: YES with windows AD

Ambari Enabled AD: YES

KNOX Enabled with AD: YES (except Admin account/topology)

KNOX advance topology have definition for zookeeper dynamic discovery and webhdfs ha: YES

While connecting on HIVE through beeline getting error as "Invalid status 72 (state=08S01,code=0)"

[ajay@vijayhdp-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayhdp-1.novalocal:8443/;ssl=true;sslTrustStore=/tmp/myLocalTrustStore.jks;trustStorePassword=password?transportMode=http;httpPath=gateway/default/hive Connecting to jdbc:hive2://vijayhdp-1.novalocal:8443/;ssl=true;sslTrustStore=/tmp/myLocalTrustStore.jks;trustStorePassword=password?transportMode=http;httpPath=gateway/default/hive Enter username for jdbc:hive2://vijayhdp-1.novalocal:8443/;ssl=true;sslTrustStore=/tmp/myLocalTrustStore.jks;trustStorePassword=password?transportMode=http;httpPath=gateway/default/hive: ajay Enter password for jdbc:hive2://vijayhdp-1.novalocal:8443/;ssl=true;sslTrustStore=/tmp/myLocalTrustStore.jks;trustStorePassword=password?transportMode=http;httpPath=gateway/default/hive: ******** 17/12/22 15:43:57 [main]: WARN jdbc.HiveConnection: Failed to connect to vijayhdp-1.novalocal:8443 Error: Could not open client transport with JDBC Uri: jdbc:hive2://vijayhdp-1.novalocal:8443/;ssl=true;sslTrustStore=/tmp/myLocalTrustStore.jks;trustStorePassword=password?transportMode=http;httpPath=gateway/default/hive: Invalid status 72 (state=08S01,code=0) 0: jdbc:hive2://vijayhdp-1.novalocal:8443/ (closed)>

Kindly suggest to fix it.

- Vijay Mishra

,

@Ryan Chapin

I am unable to connect on hive through knox using zokeeper dynamic discovery. Below details of my cluster.

HDP: 2.6

Kerberos Enabled: YES ( Windows AD 2012 R2)

Authentication: AD with LDAPS ( Windows AD 2012 R2)

Ranger Enabled: YES

RANGER Usersync and GroupSync: YES with windows AD

Ambari Enabled AD: YES

KNOX Enabled with AD: YES (except Admin account/topology)

KNOX advance topology have definition for zookeeper dynamic discovery and webhdfs ha: YES

While connecting on HIVE through beeline getting error as "Invalid status 72 (state=08S01,code=0)"

[ajay@vijayhdp-1 ~]$ beeline Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect jdbc:hive2://vijayhdp-1.novalocal:8443/;ssl=true;sslTrustStore=/tmp/myLocalTrustStore.jks;trustStorePassword=password?transportMode=http;httpPath=gateway/default/hive Connecting to jdbc:hive2://vijayhdp-1.novalocal:8443/;ssl=true;sslTrustStore=/tmp/myLocalTrustStore.jks;trustStorePassword=password?transportMode=http;httpPath=gateway/default/hive Enter username for jdbc:hive2://vijayhdp-1.novalocal:8443/;ssl=true;sslTrustStore=/tmp/myLocalTrustStore.jks;trustStorePassword=password?transportMode=http;httpPath=gateway/default/hive: ajay Enter password for jdbc:hive2://vijayhdp-1.novalocal:8443/;ssl=true;sslTrustStore=/tmp/myLocalTrustStore.jks;trustStorePassword=password?transportMode=http;httpPath=gateway/default/hive: ******** 17/12/22 15:43:57 [main]: WARN jdbc.HiveConnection: Failed to connect to vijayhdp-1.novalocal:8443 Error: Could not open client transport with JDBC Uri: jdbc:hive2://vijayhdp-1.novalocal:8443/;ssl=true;sslTrustStore=/tmp/myLocalTrustStore.jks;trustStorePassword=password?transportMode=http;httpPath=gateway/default/hive: Invalid status 72 (state=08S01,code=0) 0: jdbc:hive2://vijayhdp-1.novalocal:8443/ (closed)>

Kindly suggest to fix it.

- Vijay Mishra