Member since
02-20-2018
3
Posts
0
Kudos Received
0
Solutions
04-25-2018
12:45 PM
Hi Elif, On my case problem is not using the correct ticket. I was exporting ticket everytime and after kinit it was able to get ticket but since time to time I was not using the latest process's ticket. ============================================================================ One example below : the output of hive.keytab [root@bdw1n07 sbilgic]# klist -k -t -e hive.keytab Keytab name: FILE:hive.keytab KVNO Timestamp Principal ---- ----------------- ---------------------------------------------------------------------------------------------------------------------- 13 02/27/18 08:58:51 hive/......................................@...................................... (aes256-cts-hmac-sha1-96) 13 02/27/18 08:58:51 hive/......................................@...................................... (aes128-cts-hmac-sha1-96) 13 02/27/18 08:58:51 hive/......................................@...................................... (des3-cbc-sha1) 13 02/27/18 08:58:51 hive/......................................@...................................... (arcfour-hmac) 13 02/27/18 08:58:51 hive/......................................@...................................... (des-hmac-sha1) 13 02/27/18 08:58:51 hive/......................................@I...................................... (des-cbc-md5) ============================================================================ Clearly, the hive.keytab above has not been generated by Cloudera Manager, instead, it has been created from kadmin or kadmin.local once that happens the keytab generated by Cloudera Manager fails with the checksum. I used a copy of hive.keytab generated from Cloudera Manager copying it from the process directory. ***Not that the command: kinit -kt /var/run/cloudera-scm-agent/process/`ls -1 /var/run/cloudera-scm-agent/process | grep HIVESERVER2 | sort -n | tail -1`/hive.keytab hive/$(hostname -f) kinit with the latest process directory for hive from /var/run/cloudera-scm-agent/process/ ***the latest process directory is collected with the command below: ls -ltr /var/run/cloudera-scm-agent/process/ | grep HIVESERVER2 ***Note that the hive.keytab under the process directory /var/run/cloudera-scm-agent/process/NNN-hive-HIVESERVER2/hive.keytab Has principals for hive and HTTP once the customer has configured HiveServer2 WebUI. So, if you are doing, do not export keytab from kadmin or kadmin.local, unless you are willing to configure Hive to use that keytab. Instead get a copy of the hive.keytab from the process directory: /var/run/cloudera-scm-agent/process/NNN-hive-HIVESERVER2/hive.keytab Please let me know if you have further questions.
... View more
02-21-2018
12:19 AM
Hi, Basically, I am trying to connect hive on kerberos enabled environment. Sharing all the details that I am using: Here I have the keytab: [root@bdw1n10 keytabs]# ls -lrt
total 8
-r--r----- 1 hdfs hadoop 1024 Feb 13 13:35 http_secret
-rw------- 1 root root 496 Feb 20 13:32 hive.keytab
[root@bdw1n10 keytabs]# Here I have the valid ticket [root@bdw1n10 keytabs]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: hive/bdw1n10.bnet.luxds.net@INTBDA.BIL.COM
Valid starting Expires Service principal
02/20/18 13:32:30 02/21/18 13:32:30 krbtgt/INTBDA.BIL.COM@INTBDA.BIL.COM
renew until 02/25/18 13:32:30 Here how I am trying to connect to beeline: beeline> !connect jdbc:hive2://bdw1n10.bnet.luxds.net:10000/default;ssl=true;sslTrustStore=/opt/cloudera/security/jks/BDWCLUINT.truststore;trustStorePassword=mf2cy1fMiH6oRlcVBfWPsX5FyzzeDCdTynZQlOoxRrVcu4headReAAna1V2VxCMd;principal=hive/bdw1n10.bnet.luxds.net@INTBDA.BIL.COM oracle org.apache.hive.jdbc.HiveDriver
scan complete in 1ms
Connecting to jdbc:hive2://bdw1n10.bnet.luxds.net:10000/default;ssl=true;sslTrustStore=/opt/cloudera/security/jks/BDWCLUINT.truststore;trustStorePassword=mf2cy1fMiH6oRlcVBfWPsX5FyzzeDCdTynZQlOoxRrVcu4headReAAna1V2VxCMd;principal=hive/bdw1n10.bnet.luxds.net@INTBDA.BIL.COM
Unknown HS2 problem when communicating with Thrift server.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://bdw1n10.bnet.luxds.net:10000/default;ssl=true;sslTrustStore=/opt/cloudera/security/jks/BDWCLUINT.truststore;trustStorePassword=mf2cy1fMiH6oRlcVBfWPsX5FyzzeDCdTynZQlOoxRrVcu4headReAAna1V2VxCMd;principal=hive/bdw1n10.bnet.luxds.net@INTBDA.BIL.COM: Could not connect to bdw1n10.bnet.luxds.net on port 10000 (state=08S01,code=0)
beeline> Here is the error: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:199)
at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
... 10 more
2018-02-20 12:11:16,000 ERROR org.apache.thrift.transport.TSaslTransport: [HiveServer2-Handler-Pool: Thread-141]: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)]
at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:199)
at org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:539)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:283)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:793)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:790)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1897)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:790)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)
at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:856)
at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342)
at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285)
at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:167)
... 14 more
Caused by: KrbException: Checksum failed
at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:102)
at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:94)
at sun.security.krb5.EncryptedData.decrypt(EncryptedData.java:175)
at sun.security.krb5.KrbApReq.authenticate(KrbApReq.java:281)
at sun.security.krb5.KrbApReq.<init>(KrbApReq.java:149)
at sun.security.jgss.krb5.InitSecContextToken.<init>(InitSecContextToken.java:108)
at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:829)
... 17 more
Caused by: java.security.GeneralSecurityException: Checksum failed
at sun.security.krb5.internal.crypto.dk.AesDkCrypto.decryptCTS(AesDkCrypto.java:451)
at sun.security.krb5.internal.crypto.dk.AesDkCrypto.decrypt(AesDkCrypto.java:272)
at sun.security.krb5.internal.crypto.Aes256.decrypt(Aes256.java:76)
at sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType.decrypt(Aes256CtsHmacSha1EType.java:100)
... 23 more
2018-02-20 12:11:16,002 ERROR org.apache.thrift.server.TThreadPoolServer: [HiveServer2-Handler-Pool: Thread-141]: Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:793)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:790)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1897)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:790)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
... 10 more
2018-02-20 12:15:53,868 ERROR org.apache.thrift.server.TThreadPoolServer: [HiveServer2-Handler-Pool: Thread-146]: Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:793)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:790)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1897)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:790)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:199)
at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
... 10 more Br, Sercan
... View more
02-20-2018
01:59 AM
I have the same problem on my cluster. Tried to connect hive via beeline with below: !connect jdbc:hive2://hostname:10000/default;ssl=true;sslTrustStore=/opt/cloudera/security/jks/cm.truststore;trustStorePassword=......;principal=hive/hostname@realm oracle org.apache.hive.jdbc.HiveDriver But getting below. Unknown HS2 problem when communicating with Thrift server. Did you find a solution for this problem? Br, Sercan
... View more