Can't connect to Hive Metastore in cluster with Kerberos and TLS enabled: Peer indicated failure

nataliaking_csc — Fri, 16 Sep 2022 09:27:03 GMT

Using CDH 5.3, with Kerberos and TLS enabled when we got to testing loading data, noticed that connection to Hive Metastore fails. Cloudera Manager is not indicating any issues with principals and their keytabs. What may I be missing here?

2015-04-25 11:02:26,197 ERROR org.apache.thrift.server.TThreadPoolServer: Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed
   at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
   at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:724)
   at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:721)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:356)
   at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1622)
   at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge20S.java:721)
   at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed
   at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:199)
   at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
   at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:262)
   at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
   at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
   ... 10 more
2015-04-25 11:02:27,200 ERROR org.apache.thrift.server.TThreadPoolServer: Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed
   at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
   at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:724)
   at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:721)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:356)
   at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1622)
   at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge20S.java:721)
   at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed
   at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:199)
   at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
   at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:262)
   at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
   at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
   ... 10 more

Re: Can't connect to Hive Metastore in cluster with Kerberos and TLS enabled: Peer indicated failure

nataliaking_csc — Sat, 25 Apr 2015 15:56:34 GMT

Connected with beeline

1: jdbc:hive2://{hostname_of_hive_server}:10> select * from students;
+-----------------+---------------+---------------+--+
| students.sname  | students.age  | students.gpa  |
+-----------------+---------------+---------------+--+
+-----------------+---------------+---------------+--+
No rows selected (0.802 seconds)
1: jdbc:hive2://{hostname_of_hive_server}:10> select * from sales limit 5;
Error: java.io.IOException: java.io.IOException: HTTP status [500], message [Internal Server Error] (state=,code=0)

Re: Can't connect to Hive Metastore in cluster with Kerberos and TLS enabled: Peer indicated failure

nataliaking_csc — Sat, 25 Apr 2015 16:56:00 GMT

I was looking further into this and it appears the problem comes up when I first try loading data to newly created table, and throws:

Fetching results ran into the following error(s):

java.io.IOException: java.io.IOException: HTTP status [500], message [Internal Server Error]

Re: Can't connect to Hive Metastore in cluster with Kerberos and TLS enabled: Peer indicated failure

alpertankut — Wed, 03 Jan 2018 20:00:45 GMT

is this the solution?

question Re: Can't connect to Hive Metastore in cluster with Kerberos and TLS enabled: Peer indicated failure in Archives of Support Questions (Read Only)

Can't connect to Hive Metastore in cluster with Kerberos and TLS enabled: Peer indicated failure

Re: Can't connect to Hive Metastore in cluster with Kerberos and TLS enabled: Peer indicated failure

Re: Can't connect to Hive Metastore in cluster with Kerberos and TLS enabled: Peer indicated failure

Re: Can't connect to Hive Metastore in cluster with Kerberos and TLS enabled: Peer indicated failure