Remote beeline client from Mac to a secure CDH 5.14.0 cluster

[#1]

Hello,

My goal is provide documentation on remote access to Hive and we primarily use Mac in our department. I was able to install beeline based on the following blog, link

I know beeline is working correctly as I’m able to connect to small CDH 5.14.2 cluster that does not have Kerberos or TLS enabled.

When connecting to our Kerberos and TLS enable cluster, I get the following error message:

readlink: illegal option -- f
usage: readlink [-n] [file ...]
scan complete in 2ms
Connecting to jdbc:hive2://FQDN.HOST:10000/default;principal=hive/FQDN_HOSTNAME@REALM
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/user_name/opt/apache-hive-1.1.0-bin/lib/hive-jdbc-1.1.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/user_name/opt/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <a href="<a href="http://www.slf4j.org/codes.html#multiple_bindings" target="_blank">http://www.slf4j.org/codes.html#multiple_bindings</a>" target="_blank"><a href="http://www.slf4j.org/codes.html#multiple_bindings</a" target="_blank">http://www.slf4j.org/codes.html#multiple_bindings</a</a>> for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
19/10/17 17:06:47 [main]: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Can't get Kerberos realm
Beeline version 1.1.0 by Apache Hive
0: jdbc:hive2://FQDN.HOST:10000 (closed)>

I copied over the /etc/krb5.conf from a data node to my mac and got a different error message:

Connecting to jdbc:hive2://FQDN_HOSTNAME:10000/default;principal=hive/ FQDN_HOSTNAME@REALM
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/user_name/opt/apache-hive-1.1.0-bin/lib/hive-jdbc-1.1.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/user_name/opt/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <a href="<a href="http://www.slf4j.org/codes.html#multiple_bindings" target="_blank">http://www.slf4j.org/codes.html#multiple_bindings</a>" target="_blank"><a href="http://www.slf4j.org/codes.html#multiple_bindings</a" target="_blank">http://www.slf4j.org/codes.html#multiple_bindings</a</a>> for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
19/10/17 20:47:59 [main]: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Error: Could not open client transport with JDBC Uri: jdbc:hive2://FQDN_HOSTNAME:10000/default;principal=hive/FQDN_HOSTNAME@REALM: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0)
Beeline version 1.1.0 by Apache Hive
0: jdbc:hive2://FQDN_HOSTNAME:10000 (closed)>
0: jdbc:hive2://FQDN_HOSTNAME:10000 (closed)> show databases;
19/10/17 20:48:31 [main]: ERROR transport.TSaslTransport: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

Thank you for your time, 

[#2]

Hi,

        Please export the KRB5_CONFIG path in shell and perform kinit as a valid user before running the command.  

export KRB5_CONFIG=/etc/krb5.conf

kinit  - as a valid user..

Run your command.

Please try and share the results.

Regards,

Ganesh

[#3]

Hi Ganesh,

I forgot to mention that my mac does have an active Kerberos ticket for my user account. Here is the output of the test.

mac_laptop:opt user_name$ export KRB5_CONFIG=/etc/krb5.conf
mac_laptop:opt user_name$ kdestroy
mac_laptop:opt user_name$ kinit
user1@CORP_DOMAIN password:
mac_laptop:opt user_name$ klist
Credentials cache: API:5CA7B7AE-1F94-4275-96A7-33EA33E17D38
        Principal: uuser_name@CORP_DOMAIN

  Issued                Expires               Principal
Oct 18 07:42:49 2019  Oct 18 17:42:43 2019  krbtgt/CORP_DOMAIN@CORP_DOMAIN
mac_laptop:opt user_name$ beeline -u "jdbc:hive2://FQDN_HOSTNAME:10000/default;principal=hive/FQDN_HOSTNAME@REALM"
readlink: illegal option -- f
usage: readlink [-n] [file ...]
scan complete in 3ms
Connecting to jdbc:hive2://FQDN_HOSTNAME:10000/default;principal=hive/FQDN_HOSTNAME@REALM
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/user_name/opt/apache-hive-1.1.0-bin/lib/hive-jdbc-1.1.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/user_name/opt/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <a href="http://www.slf4j.org/codes.html#multiple_bindings" target="_blank">http://www.slf4j.org/codes.html#multiple_bindings</a> for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
19/10/18 07:43:05 [main]: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Error: Could not open client transport with JDBC Uri: jdbc:hive2://FQDN_HOSTNAME:10000/default;principal=hive/FQDN_HOSTNAME@REALM: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0)
Beeline version 1.1.0 by Apache Hive
0: jdbc:hive2://FQDN_HOSTNAME:10000 (closed)>

Don't Know if this is important, but the CDH cluster is using a MIT KDC and our user accounts are in the corporate domain using Active Directory. We have a one-way trust established. I'm able to connect with Beeline from a CDH cluster node. When connecting with Beeline I can connect with a Kerberos ticket from the MIT KDC and from the corporate Active Directory. I'm using the same Active Directory user account to test connecting from my mac that I've successfully connected through beeline on a CDH cluster node. 

Thank you for the help!

[#4 · Accepted solution]

Hi all,

Figured it out and got it working. In addition to needing the /etc/krb5.conf I also needed the hive-site.xml and core-site.xml from the secure CDH cluster. Copied both hive-site.xml and core-site.xml to my local apache-hive-X-bin/conf/ directory.

Thanks,