Hello, I am using secured hive streaming , in a cluster with Kerberos Authentication. I use hadoop : 2.7.3.2.6.1.0-129 hive : 1.2.1000.2.6.1.0-129 I am able to get the Ugi authenticated with my kerberos id and keytab at ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI("user@XX.YYYY.NET", "/home/xx/user.keytab"); but later when the fetchTransactionBatch is called, Kerberos IO failure occurs at TransactionBatch txnBatch = secureConn.fetchTransactionBatch(numTrx, writer); code snippet: HiveEndPoint hiveEP = new HiveEndPoint(hiveConf.getVar(ConfVars.METASTOREURIS), database, table, partArray);
Configuration conf = new Configuration();
conf.set("hadoop.security.authentication", "kerberos");
System.setProperty( "java.security.krb5.conf", "//etc//krb5.conf");
UserGroupInformation ugi;
UserGroupInformation.setConfiguration(conf);
ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI("user@XX.YYYY.NET", "/home/xx/user.keytab");
if (ugi==null)
{
return;
}
UserGroupInformation.setLoginUser(ugi);
StreamingConnection secureConn = hiveEP.newConnection(true, hiveConf, ugi);
int numTrx = 2;
String[] fieldNames = {"number", "name"};
boolean ifc = ugi.hasKerberosCredentials();
DelimitedInputWriter writer = new DelimitedInputWriter(fieldNames,",", hiveEP);
TransactionBatch txnBatch = secureConn.fetchTransactionBatch(numTrx, writer); Exception at : TransactionBatch txnBatch = secureConn.fetchTransactionBatch(numTrx, writer); java.lang.reflect.UndeclaredThrowableException
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884)
at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.fetchTransactionBatch(HiveEndPoint.java:424)
at ns.KClass.<init>(KClass.java:112)
at ns.KClass.main(KClass.java:45)
Caused by: org.apache.hive.hcatalog.streaming.StreamingIOFailure: Failed creating RecordUpdaterS for hdfs://MAIN/hdfs/path/to/database.db/acid1 txnIds[113,114]
at org.apache.hive.hcatalog.streaming.AbstractRecordWriter.newBatch(AbstractRecordWriter.java:193) caused by: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "lh.net/1.2.3.4"; destination host is: "yyy.net":8020; Can someone tell me how to solve this exception and get the Transaction batch. Thanks.
... View more
