Secured Hive Streaming Kerberos Exception

[#1]

Hello,

I am using secured hive streaming , in a cluster with Kerberos Authentication.

I use hadoop : 2.7.3.2.6.1.0-129

hive : 1.2.1000.2.6.1.0-129

I am able to get the Ugi authenticated with my kerberos id and keytab at

ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI("user@XX.YYYY.NET", "/home/xx/user.keytab");

but later when the fetchTransactionBatch is called, Kerberos IO failure occurs at

TransactionBatch txnBatch = secureConn.fetchTransactionBatch(numTrx, writer);

code snippet:

HiveEndPoint hiveEP = new HiveEndPoint(hiveConf.getVar(ConfVars.METASTOREURIS), database, table, partArray);
Configuration conf = new Configuration();
conf.set("hadoop.security.authentication", "kerberos");
System.setProperty( "java.security.krb5.conf", "//etc//krb5.conf");
UserGroupInformation ugi;
UserGroupInformation.setConfiguration(conf);
ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI("user@XX.YYYY.NET", "/home/xx/user.keytab");
if (ugi==null)
{
return;
}
UserGroupInformation.setLoginUser(ugi);
StreamingConnection secureConn = hiveEP.newConnection(true, hiveConf, ugi);
int numTrx = 2;
String[] fieldNames = {"number", "name"};
boolean ifc = ugi.hasKerberosCredentials();
DelimitedInputWriter writer = new DelimitedInputWriter(fieldNames,",", hiveEP);
TransactionBatch txnBatch = secureConn.fetchTransactionBatch(numTrx, writer);

Exception at : TransactionBatch txnBatch = secureConn.fetchTransactionBatch(numTrx, writer);

java.lang.reflect.UndeclaredThrowableException at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884) at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.fetchTransactionBatch(HiveEndPoint.java:424) at ns.KClass.<init>(KClass.java:112) at ns.KClass.main(KClass.java:45) Caused by: org.apache.hive.hcatalog.streaming.StreamingIOFailure: Failed creating RecordUpdaterS for hdfs://MAIN/hdfs/path/to/database.db/acid1 txnIds[113,114] at org.apache.hive.hcatalog.streaming.AbstractRecordWriter.newBatch(AbstractRecordWriter.java:193)

caused by: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "lh.net/1.2.3.4"; destination host is: "yyy.net":8020;

Can someone tell me how to solve this exception and get the Transaction batch.

Thanks.

[#2]

@Opao E

If I understand correctly, you need to provide kerberos principal for hive in your UGI call and then pass your "user@XX.MADM.NET" as proxy.

UserGroupInformation ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI (configurationService.kerberos_principal, configurationService.kerberos_keytab);
//kerberos principal here is:
/* <property>
  <name>hive.server2.authentication.kerberos.principal</name>
  <value>hive/_HOST@YOUR-REALM.COM</value>
</property>
*/
Class.forName(configurationService.hive_JDBC_DRIVER);

String uri = configurationService.hive_JDBC_DB_URL + "/" + db + ";principal=" +
configurationService.kerberos_hiveServer2Principal
+ ";hive.server2.proxy.user=" + UserUtil.getUserName(); //replace UserUtil.getUserName() with your username.