question Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN in Archives of Support Questions (Read Only)

Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

geko — Sun, 20 Dec 2015 03:15:06 GMT

Hi,

I am running a fresh installed HDP cluster with Kerberos enabled.

I try to connect to Hive using beeline command:

beeline -u jdbc:hive2://deala01876.corp:10000/default;principal=hive/deala01876.corp@HDP.REALM

I receive the following error:

scan complete in 5ms
Connecting to jdbc:hive2://deala01876.corp:10000/default
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings... for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Error: Could not open client transport with JDBC Uri: jdbc:hive2://deala01876.corp:10000/default: Peer indicated failure: Unsupported mechanism type PLAIN (state=08S01,code=0)
Beeline version 0.14.0.2.2.4.2-2 by Apache Hive
0: jdbc:hive2://deala01876.corp:1 (closed)>

The local user has a valid Kerberos ticket, as well as the hive user on the Hiveserver node.

HiveServer2 authentication is set to "Kerberos", and property "hive.server2.authentication.kerberos.principal" is set to value " hive/_HOST@HDP.REALM"

What is going wrong here, what to check further ?

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

nsabharwal — Sun, 20 Dec 2015 07:46:41 GMT

@Gerd Koenig please paste output of klist

make sure that you have correct/valid ticket.

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

geko — Sun, 20 Dec 2015 19:02:29 GMT

Hi @Neeraj Sabharwal ,

here the details from the client node, from which I want to execute beeline:

W999711@DEALA01885:~$ kinit -kt /etc/security/keytabs/w999711.user.keytab w999711
W999711@DEALA01885:~$ klist
Ticket cache: FILE:/tmp/krb5cc_2001012
Default principal: w999711@HDP.REALM
Valid starting     Expires            Service principal
12/20/15 11:48:17  12/21/15 11:48:17  krbtgt/HDP.REALM@HDP.REALM
        renew until 12/20/15 11:48:17
W999711@DEALA01885:~$ beeline -u jdbc:hive2://deala01876.corp:10000/default;principal=hive/deala01876.corp@HDP.REALM
scan complete in 5ms
Connecting to jdbc:hive2://deala01876.corp:10000/default
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Error: Could not open client transport with JDBC Uri: jdbc:hive2://deala01876.corp:10000/default: Peer indicated failure: Unsupported mechanism type PLAIN (state=08S01,code=0)
Beeline version 0.14.0.2.2.4.2-2 by Apache Hive
0: jdbc:hive2://deala01876.corp:1 (closed)>

and here the details from the node running Hiveserver2:

$ klist
Ticket cache: FILE:/tmp/krb5cc_16940
Default principal: hive/deala01876.corp@HDP.REALM
Valid starting     Expires            Service principal
12/19/15 20:46:28  12/20/15 20:46:28  krbtgt/HDP.REALM@HDP.REALM
        renew until 12/19/15 20:46:28
hive@DEALA01876:/home/hive 0
$ tail -f /var/log/hadooplogs/hive/hiveserver2.log
...
2015-12-20 11:53:03,660 ERROR [HiveServer2-Handler-Pool: Thread-56]: server.TThreadPoolServer (TThreadPoolServer.java:run(215)) - Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Unsupported mechanism type PLAIN
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:180)
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:726)
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:723)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:356)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1608)
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge20S.java:723)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException: Unsupported mechanism type PLAIN
        at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
        at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:138)
        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
        at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
        at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:177)
        ... 10 more

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

nsabharwal — Sun, 20 Dec 2015 21:14:43 GMT

@Gerd Koenig

In the hiveserver2 server, Get the ticket using hive keytab and then try to login to beeline. Paste the output of klist once you get the ticket

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

geko — Sun, 20 Dec 2015 22:38:45 GMT

Hi @Neeraj Sabharwal ,

on the Hiveserver2 server I have no keytab available for my personal user-id (w999711) with which I want to create a beeline connection. Therefore I tried to connect as (OS-)user 'hive' via beeline, but receive the same error message:

$ kdestroy
hive@DEALA01876:/home/hive 0
$ klist
klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_16940)
hive@DEALA01876:/home/hive 1
$ kinit -kt /etc/security/keytabs/hive.service.keytab hive/deala01876.corp
hive@DEALA01876:/home/hive 0
$ klist
Ticket cache: FILE:/tmp/krb5cc_16940
Default principal: hive/deala01876.corp@HDP.REALM
Valid starting     Expires            Service principal
12/20/15 15:29:05  12/21/15 15:29:05  krbtgt/HDP.REALM@HDP.REALM
        renew until 12/20/15 15:29:05
hive@DEALA01876:/home/hive 0
$ beeline -u jdbc:hive2://deala01876.corp:10000/default;principal=hive/deala01876.corp@HDP.REALM
scan complete in 5ms
Connecting to jdbc:hive2://deala01876.corp:10000/default
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings. for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Error: Could not open client transport with JDBC Uri: jdbc:hive2://deala01876.corp:10000/default: Peer indicated failure: Unsupported mechanism type PLAIN (state=08S01,code=0)
Beeline version 0.14.0.2.2.4.2-2 by Apache Hive
0: jdbc:hive2://deala01876.corp:1 (closed)>

..and below the corresponding config values from /etc/hive/conf.server/

/etc/hive/conf.server/hive-site.xml-    <property>
/etc/hive/conf.server/hive-site.xml:      <name>hive.server2.authentication</name>
/etc/hive/conf.server/hive-site.xml-      <value>KERBEROS</value>
/etc/hive/conf.server/hive-site.xml-    </property>
/etc/hive/conf.server/hive-site.xml-
/etc/hive/conf.server/hive-site.xml-    <property>
/etc/hive/conf.server/hive-site.xml:      <name>hive.server2.authentication.kerberos.keytab</name>
/etc/hive/conf.server/hive-site.xml-      <value>/etc/security/keytabs/hive.service.keytab</value>
/etc/hive/conf.server/hive-site.xml-    </property>
/etc/hive/conf.server/hive-site.xml-
/etc/hive/conf.server/hive-site.xml-    <property>
/etc/hive/conf.server/hive-site.xml:      <name>hive.server2.authentication.kerberos.principal</name>
/etc/hive/conf.server/hive-site.xml-      <value>hive/_HOST@HDP.REALM</value>
/etc/hive/conf.server/hive-site.xml-    </property>
/etc/hive/conf.server/hive-site.xml-
/etc/hive/conf.server/hive-site.xml-    <property>
/etc/hive/conf.server/hive-site.xml:      <name>hive.server2.authentication.spnego.keytab</name>
/etc/hive/conf.server/hive-site.xml-      <value>/etc/security/keytabs/spnego.service.keytab</value>
/etc/hive/conf.server/hive-site.xml-    </property>
/etc/hive/conf.server/hive-site.xml-
/etc/hive/conf.server/hive-site.xml-    <property>
/etc/hive/conf.server/hive-site.xml:      <name>hive.server2.authentication.spnego.principal</name>
/etc/hive/conf.server/hive-site.xml-      <value>HTTP/_HOST@HDP.REALM</value>
/etc/hive/conf.server/hive-site.xml-    </property>
/etc/hive/conf.server/hive-site.xml-
/etc/hive/conf.server/hive-site.xml-    <property>
/etc/hive/conf.server/hive-site.xml:      <name>hive.server2.enable.doAs</name>
/etc/hive/conf.server/hive-site.xml-      <value>true</value>
/etc/hive/conf.server/hive-site.xml-    </property>

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

VR46 — Mon, 21 Dec 2015 15:15:37 GMT

Hi @Gerd Koenig ,

From the beeline log, it is clear that the beeline is still not able to understand that you want to communication over Kerberos. Can you please post the output of same the beeline command with '-v' flag?

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

geko — Mon, 21 Dec 2015 15:44:26 GMT

Hi @Vipin Rathor,

thanks for jumping in 😄 . Please find the output below (unfortunately not that meaningful.....):

W999711@DEALA01885:~$ beeline --verbose=true -u jdbc:hive2://deala01876.corp:10000/default;principal=hive/_HOST@HDP.REALM                
issuing: !connect jdbc:hive2://deala01876.corp:10000/default '' ''
scan complete in 5ms
Connecting to jdbc:hive2://deala01876.corp:10000/default
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings... for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Error: Could not open client transport with JDBC Uri: jdbc:hive2://deala01876.corp:10000/default: Peer indicated failure: Unsupported mechanism type PLAIN (state=08S01,code=0)
java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://deala01876.corp:10000/default: Peer indicated failure: Unsupported mechanism type PLAIN
        at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215)
        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
        at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:187)
        at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:138)
        at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:179)
        at org.apache.hive.beeline.Commands.connect(Commands.java:1078)
        at org.apache.hive.beeline.Commands.connect(Commands.java:999)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:45)
        at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:936)
        at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:698)
        at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:748)
        at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:476)
        at org.apache.hive.beeline.BeeLine.main(BeeLine.java:459)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.thrift.transport.TTransportException: Peer indicated failure: Unsupported mechanism type PLAIN
        at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:190)
        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:288)
        at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
        at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190)
        ... 24 more
Beeline version 0.14.0.2.2.4.2-2 by Apache Hive
0: jdbc:hive2://deala01876.corp:1 (closed)>

...and the error message is the same, either using "_HOST" or the real hostname "deala01876.corp" in the principal.

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

geko — Mon, 21 Dec 2015 17:54:37 GMT

@Vipin Rathor , @Neeraj Sabharwal,

it is solved.....and what a stupid cause 😉

The connection can be established if I put quotes around the JDBC URL =>

W999711@DEALA01885:~$ beeline -u "jdbc:hive2://deala01876.corp:10000/default;principal=hive/deala01876.corp@HDP.REALM"
scan complete in 5ms
Connecting to jdbc:hive2://deala01876.corp:10000/default;principal=hive/deala01876.corp@HDP.REALM
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings. for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Connected to: Apache Hive (version 0.14.0.2.2.4.2-2)
Driver: Hive JDBC (version 0.14.0.2.2.4.2-2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 0.14.0.2.2.4.2-2 by Apache Hive
0: jdbc:hive2://deala01876.corp:1>

sorry for causing any confusion 😄

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

nsabharwal — Mon, 21 Dec 2015 17:55:19 GMT

@Gerd Koenig That's :))

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

margusja — Sun, 10 Jan 2016 16:31:43 GMT

try beeline --verbose=true -u "jdbc:hive2://deala01876.corp:10000/default;principal=hive/_HOST@HDP.REALM"

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

murari_goswami — Tue, 12 Jan 2016 21:36:02 GMT

The one liner :

beeline -u "jdbc:hive2://ip-hive-server-ip separated with hifen>:10000/default;principal=driver-details" -n $USER

Re: Beeline connection fails with: Peer indicated failure: Unsupported mechanism type PLAIN

sohaib1692 — Tue, 23 Feb 2016 16:26:57 GMT

Did you figure out why this happens though? Is @ a special character?