Hi @vperiasamy , yes, the user 'hdfs' has been sync'ed to Ranger and he is part of that group on OS level (the hadoop-admins group exists on all nodes in the cluster and 'hdfs' is member on all nodes as well). => user is there, but the group not.... Any hint highly appreciated 😉 ... View more

Hi @Vipin Rathor, thanks for jumping in 😄 . Please find the output below (unfortunately not that meaningful.....): W999711@DEALA01885:~$ beeline --verbose=true -u jdbc:hive2://deala01876.corp:10000/default;principal=hive/_HOST@HDP.REALM issuing: !connect jdbc:hive2://deala01876.corp:10000/default '' '' scan complete in 5ms Connecting to jdbc:hive2://deala01876.corp:10000/default SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings... for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Error: Could not open client transport with JDBC Uri: jdbc:hive2://deala01876.corp:10000/default: Peer indicated failure: Unsupported mechanism type PLAIN (state=08S01,code=0) java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://deala01876.corp:10000/default: Peer indicated failure: Unsupported mechanism type PLAIN at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:187) at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:138) at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:179) at org.apache.hive.beeline.Commands.connect(Commands.java:1078) at org.apache.hive.beeline.Commands.connect(Commands.java:999) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:45) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:936) at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:698) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:748) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:476) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:459) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: org.apache.thrift.transport.TTransportException: Peer indicated failure: Unsupported mechanism type PLAIN at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:190) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:288) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190) ... 24 more Beeline version 0.14.0.2.2.4.2-2 by Apache Hive 0: jdbc:hive2://deala01876.corp:1 (closed)> ...and the error message is the same, either using "_HOST" or the real hostname "deala01876.corp" in the principal. ... View more

Hi @Neeraj Sabharwal , on the Hiveserver2 server I have no keytab available for my personal user-id (w999711) with which I want to create a beeline connection. Therefore I tried to connect as (OS-)user 'hive' via beeline, but receive the same error message: $ kdestroy hive@DEALA01876:/home/hive 0 $ klist klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_16940) hive@DEALA01876:/home/hive 1 $ kinit -kt /etc/security/keytabs/hive.service.keytab hive/deala01876.corp hive@DEALA01876:/home/hive 0 $ klist Ticket cache: FILE:/tmp/krb5cc_16940 Default principal: hive/deala01876.corp@HDP.REALM Valid starting Expires Service principal 12/20/15 15:29:05 12/21/15 15:29:05 krbtgt/HDP.REALM@HDP.REALM renew until 12/20/15 15:29:05 hive@DEALA01876:/home/hive 0 $ beeline -u jdbc:hive2://deala01876.corp:10000/default;principal=hive/deala01876.corp@HDP.REALM scan complete in 5ms Connecting to jdbc:hive2://deala01876.corp:10000/default SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings. for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Error: Could not open client transport with JDBC Uri: jdbc:hive2://deala01876.corp:10000/default: Peer indicated failure: Unsupported mechanism type PLAIN (state=08S01,code=0) Beeline version 0.14.0.2.2.4.2-2 by Apache Hive 0: jdbc:hive2://deala01876.corp:1 (closed)> ..and below the corresponding config values from /etc/hive/conf.server/ /etc/hive/conf.server/hive-site.xml- <property> /etc/hive/conf.server/hive-site.xml: <name>hive.server2.authentication</name> /etc/hive/conf.server/hive-site.xml- <value>KERBEROS</value> /etc/hive/conf.server/hive-site.xml- </property> /etc/hive/conf.server/hive-site.xml- /etc/hive/conf.server/hive-site.xml- <property> /etc/hive/conf.server/hive-site.xml: <name>hive.server2.authentication.kerberos.keytab</name> /etc/hive/conf.server/hive-site.xml- <value>/etc/security/keytabs/hive.service.keytab</value> /etc/hive/conf.server/hive-site.xml- </property> /etc/hive/conf.server/hive-site.xml- /etc/hive/conf.server/hive-site.xml- <property> /etc/hive/conf.server/hive-site.xml: <name>hive.server2.authentication.kerberos.principal</name> /etc/hive/conf.server/hive-site.xml- <value>hive/_HOST@HDP.REALM</value> /etc/hive/conf.server/hive-site.xml- </property> /etc/hive/conf.server/hive-site.xml- /etc/hive/conf.server/hive-site.xml- <property> /etc/hive/conf.server/hive-site.xml: <name>hive.server2.authentication.spnego.keytab</name> /etc/hive/conf.server/hive-site.xml- <value>/etc/security/keytabs/spnego.service.keytab</value> /etc/hive/conf.server/hive-site.xml- </property> /etc/hive/conf.server/hive-site.xml- /etc/hive/conf.server/hive-site.xml- <property> /etc/hive/conf.server/hive-site.xml: <name>hive.server2.authentication.spnego.principal</name> /etc/hive/conf.server/hive-site.xml- <value>HTTP/_HOST@HDP.REALM</value> /etc/hive/conf.server/hive-site.xml- </property> /etc/hive/conf.server/hive-site.xml- /etc/hive/conf.server/hive-site.xml- <property> /etc/hive/conf.server/hive-site.xml: <name>hive.server2.enable.doAs</name> /etc/hive/conf.server/hive-site.xml- <value>true</value> /etc/hive/conf.server/hive-site.xml- </property> ... View more

Hi @Neeraj Sabharwal , here the details from the client node, from which I want to execute beeline: W999711@DEALA01885:~$ kinit -kt /etc/security/keytabs/w999711.user.keytab w999711 W999711@DEALA01885:~$ klist Ticket cache: FILE:/tmp/krb5cc_2001012 Default principal: w999711@HDP.REALM Valid starting Expires Service principal 12/20/15 11:48:17 12/21/15 11:48:17 krbtgt/HDP.REALM@HDP.REALM renew until 12/20/15 11:48:17 W999711@DEALA01885:~$ beeline -u jdbc:hive2://deala01876.corp:10000/default;principal=hive/deala01876.corp@HDP.REALM scan complete in 5ms Connecting to jdbc:hive2://deala01876.corp:10000/default SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Error: Could not open client transport with JDBC Uri: jdbc:hive2://deala01876.corp:10000/default: Peer indicated failure: Unsupported mechanism type PLAIN (state=08S01,code=0) Beeline version 0.14.0.2.2.4.2-2 by Apache Hive 0: jdbc:hive2://deala01876.corp:1 (closed)> and here the details from the node running Hiveserver2: $ klist Ticket cache: FILE:/tmp/krb5cc_16940 Default principal: hive/deala01876.corp@HDP.REALM Valid starting Expires Service principal 12/19/15 20:46:28 12/20/15 20:46:28 krbtgt/HDP.REALM@HDP.REALM renew until 12/19/15 20:46:28 hive@DEALA01876:/home/hive 0 $ tail -f /var/log/hadooplogs/hive/hiveserver2.log ... 2015-12-20 11:53:03,660 ERROR [HiveServer2-Handler-Pool: Thread-56]: server.TThreadPoolServer (TThreadPoolServer.java:run(215)) - Error occurred during processing of message. java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Unsupported mechanism type PLAIN at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:180) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:726) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:723) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:356) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1608) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge20S.java:723) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.thrift.transport.TTransportException: Unsupported mechanism type PLAIN at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221) at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:138) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253) at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:177) ... 10 more ... View more

Thanks @Neeraj Sabharwal You are not concerned about having different values for "Enable authorization" before enabling Ranger 😉 ? ... View more

Hello @sneethiraj , thanks for your answer. On all nodes the group hadoop-admins contains: hadoop-admins:x:23231:w999711,w1004360,hdfs and at least the user 'hdfs' is a local user and already sync'ed into Ranger, and I also restarted the ranger-usersync service with the following log entries: 18 Dec 2015 08:42:00 INFO UnixAuthenticationService [main] - Starting User Sync Service! 18 Dec 2015 08:42:00 INFO UnixAuthenticationService [main] - Enabling Unix Auth Service! 18 Dec 2015 08:42:00 INFO UserGroupSync [UnixUserSyncThread] - initializing sink: com.xasecure.unixusersync.process.PolicyMgrUserGroupBuilder 18 Dec 2015 08:42:00 INFO UnixAuthenticationService [main] - Enabling Protocol: [SSLv2Hello] 18 Dec 2015 08:42:00 INFO UnixAuthenticationService [main] - Enabling Protocol: [TLSv1] 18 Dec 2015 08:42:00 INFO UnixAuthenticationService [main] - Enabling Protocol: [TLSv1.1] 18 Dec 2015 08:42:00 INFO UnixAuthenticationService [main] - Enabling Protocol: [TLSv1.2] 18 Dec 2015 08:42:01 INFO UserGroupSync [UnixUserSyncThread] - initializing source: com.xasecure.unixusersync.process.UnixUserGroupBuilder 18 Dec 2015 08:42:01 INFO UserGroupSync [UnixUserSyncThread] - Begin: initial load of user/group from source==>sink 18 Dec 2015 08:42:01 INFO UserGroupSync [UnixUserSyncThread] - End: initial load of user/group from source==>sink 18 Dec 2015 08:42:01 INFO UserGroupSync [UnixUserSyncThread] - Done initializing user/group source and sink But I still cannot see the group hadoop-admins in Ranger: What else to check ? Is there some Debug output possible for usersync process ?!? ... View more