Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Connecting to hiveserver2 on kerberized cluster as non-hive user returns "Peer indicated failure: GSS initiate failed"

Highlighted

Connecting to hiveserver2 on kerberized cluster as non-hive user returns "Peer indicated failure: GSS initiate failed"

New Contributor

Hello

I am on Ambari 2.7 and HDP 3.1. Keberos enabled.

I have been trying to connect to HS2 (beeline) as non-hive user (test user). Here is the sample JDBC string I pass along with the output I get:

1: jdbc:hive2://ip-10-215-17-107.ec2.internal> !connect jdbc:hive2://ip-10-215-17-107.ec2.internal:10000/default;principal=test/ip-10-215-17-107.ec2.interna...
Connecting to jdbc:hive2://ip-10-215-17-107.ec2.internal:10000/default;principal=test/ip-10-215-17-107.ec2.internal@SUPPORT.COM
19/07/03 16:12:50 [main]: WARN jdbc.HiveConnection: Failed to connect to ip-10-215-17-107.ec2.internal:10000
Unknown HS2 problem when communicating with Thrift server.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://ip-10-215-17-107.ec2.internal:10000/default;principal=test/ip-10-215-17-107.ec2.internal@SUPPORT.COM: Peer indicated failure: GSS initiate failed (state=08S01,code=0)


When doing the same for hive user all works fine:

1: jdbc:hive2://ip-10-215-17-107.ec2.internal> !connect jdbc:hive2://ip-10-215-17-107.ec2.internal:10000/default;principal=hive/ip-10-215-17-107.ec2.internal@SUPPORT.COM
Connecting to jdbc:hive2://ip-10-215-17-107.ec2.internal:10000/default;principal=hive/ip-10-215-17-107.ec2.internal@SUPPORT.COM
Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
Transaction isolation: TRANSACTION_REPEATABLE_READ


Hiveserver2 log for test user:

2019-07-03T16:20:34,600 ERROR [HiveServer2-Handler-Pool: Thread-78]: transport.TSaslTransport (:()) - SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed
    at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:199) ~[?:1.8.0_112]
    at org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:539) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:283) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
    at javax.security.auth.Subject.doAs(Subject.java:360) ~[?:1.8.0_112]
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) ~[hadoop-common-3.1.1.3.1.0.0-78.jar:?]
    at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_112]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_112]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: org.ietf.jgss.GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)
    at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:856) ~[?:1.8.0_112]
    at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342) ~[?:1.8.0_112]
    at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285) ~[?:1.8.0_112]
    at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:167) ~[?:1.8.0_112]
    ... 14 more
Caused by: sun.security.krb5.KrbCryptoException: Checksum failed
    at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:102) ~[?:1.8.0_112]
    at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:94) ~[?:1.8.0_112]
    at sun.security.krb5.EncryptedData.decrypt(EncryptedData.java:175) ~[?:1.8.0_112]
    at sun.security.krb5.KrbApReq.authenticate(KrbApReq.java:281) ~[?:1.8.0_112]
    at sun.security.krb5.KrbApReq.<init>(KrbApReq.java:149) ~[?:1.8.0_112]
    at sun.security.jgss.krb5.InitSecContextToken.<init>(InitSecContextToken.java:108) ~[?:1.8.0_112]
    at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:829) ~[?:1.8.0_112]
    at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342) ~[?:1.8.0_112]
    at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285) ~[?:1.8.0_112]
    at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:167) ~[?:1.8.0_112]
    ... 14 more
Caused by: java.security.GeneralSecurityException: Checksum failed
    at sun.security.krb5.internal.crypto.dk.AesDkCrypto.decryptCTS(AesDkCrypto.java:451) ~[?:1.8.0_112]
    at sun.security.krb5.internal.crypto.dk.AesDkCrypto.decrypt(AesDkCrypto.java:272) ~[?:1.8.0_112]
    at sun.security.krb5.internal.crypto.Aes256.decrypt(Aes256.java:76) ~[?:1.8.0_112]
    at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:100) ~[?:1.8.0_112]
    at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:94) ~[?:1.8.0_112]
    at sun.security.krb5.EncryptedData.decrypt(EncryptedData.java:175) ~[?:1.8.0_112]
    at sun.security.krb5.KrbApReq.authenticate(KrbApReq.java:281) ~[?:1.8.0_112]
    at sun.security.krb5.KrbApReq.<init>(KrbApReq.java:149) ~[?:1.8.0_112]
    at sun.security.jgss.krb5.InitSecContextToken.<init>(InitSecContextToken.java:108) ~[?:1.8.0_112]
    at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:829) ~[?:1.8.0_112]
    at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342) ~[?:1.8.0_112]
    at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285) ~[?:1.8.0_112]
    at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:167) ~[?:1.8.0_112]
    ... 14 more
2019-07-03T16:20:34,600 ERROR [HiveServer2-Handler-Pool: Thread-78]: server.TThreadPoolServer (:()) - Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: GSS initiate failed
    at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:694) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:691) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
    at javax.security.auth.Subject.doAs(Subject.java:360) ~[?:1.8.0_112]
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710) ~[hadoop-common-3.1.1.3.1.0.0-78.jar:?]
    at org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:691) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_112]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_112]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
    at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) ~[hive-exec-3.1.0.3.1.0.0-78.jar:3.1.0.3.1.0.0-78]
    ... 10 more


I have keytab for test user and TGT ticket generated. But still the same issue.

Also tried with

jdbc:hive2://ip-10-215-17-14.ec2.internal:2181,ip-10-215-17-107.ec2.internal:2181,ip-10-215-19-2.ec2.internal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=test/ip-10-215-17-107.ec2.internal@SUPPORT.COM

but no success.

Any pointers much appreciated. Thanks

Don't have an account?
Coming from Hortonworks? Activate your account here