Created on 12-07-2017 05:33 AM - edited 09-16-2022 05:36 AM
this problem bother me long time, please help me!
hbase cluster with kerberos works well.
my java code for anthentication as below
static { System.setProperty("java.security.krb5.conf",DeviceChannelHbaseClient.class.getResource("/krb5.conf").getPath()); UserGroupInformation.setConfiguration(HBASE_CONF); try { URL keyTabUrl = DeviceChannelHbaseClient.class.getResource("/hbase-rw.keytab"); if (keyTabUrl != null) { UserGroupInformation.loginUserFromKeytab("hbase-rw/hz@HZ.NETEASE.COM", DeviceChannelHbaseClient.class.getResource("/hbase-rw.keytab").getPath()); } } catch (IOException e) { e.printStackTrace(); } }
at the starting, everythings works, but after a few days, an exception accurs:
[WARN ]16:49:48,007, [Class]RpcClientImpl, Exception encountered while connecting to the server : java.lang.IllegalStateException: This ticket is no longer valid [WARN ]16:49:48,007, [Class]UserGroupInformation, PriviledgedActionException as:hbase-rw/hz@HZ.CLOUD.COM (auth:KERBEROS) cause:java.io.IOExcept ion: java.lang.IllegalStateException: This ticket is no longer valid [WARN ]16:49:50,131, [Class]RpcClientImpl, Exception encountered while connecting to the server : java.lang.IllegalStateException: This ticket is no longer valid [WARN ]16:49:50,131, [Class]UserGroupInformation, PriviledgedActionException as:hbase-rw/hz@HZ.CLOUD.COM (auth:KERBEROS) cause:java.io.IOExcept ion: java.lang.IllegalStateException: This ticket is no longer valid [WARN ]16:49:52,154, [Class]UserGroupInformation, PriviledgedActionException as:hbase-rw/hz@HZ.CLOUD.COM (auth:KERBEROS) cause:javax.security.s asl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [WARN ]16:49:52,154, [Class]RpcClientImpl, Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] [FATAL]16:49:52,165, [Class]RpcClientImpl, SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'. javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find an y Kerberos tgt)] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:181) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:618) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:163) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:744) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:741) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:741) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:907) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:874) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1243) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:34070) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1594) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1398) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1199) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1166) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1150) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:971) at org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83) at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:79) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:930) at org.apache.hadoop.hbase.client.HTable.exists(HTable.java:1436) at com.netease.yx.hbase.LMRegistUserHbase.exists(LMRegistUserHbase.java:270) at com.netease.yx.leave.motivate.LMLogConsumer$TestConsumeRunner.dealWebLeaveLog(LMLogConsumer.java:132) at com.netease.yx.leave.motivate.LMLogConsumer$TestConsumeRunner.run(LMLogConsumer.java:268) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt) at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121) at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:34070) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1594) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1398) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1199) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1166) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1150) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:971) at org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83) at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:79) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:930) at org.apache.hadoop.hbase.client.HTable.exists(HTable.java:1436) at com.netease.yx.hbase.LMRegistUserHbase.exists(LMRegistUserHbase.java:270) at com.netease.yx.leave.motivate.LMLogConsumer$TestConsumeRunner.dealWebLeaveLog(LMLogConsumer.java:132) at com.netease.yx.leave.motivate.LMLogConsumer$TestConsumeRunner.run(LMLogConsumer.java:268) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt) at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121) at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ... 32 more
why? please help me , thanks very much.
Created 12-08-2017 03:09 AM
Hello,
The ticket you acquire from the keytab has an expiry date and a max renewable date.
So, if you see that error after a few days, it might just be that (either the expiry date or the max renewable date).
You need to "handle" these cases.
regards,
Mathieu
Created 12-13-2017 12:05 AM
I think the keytab you used has expired.
Try to kinit a new keytab for your code, and issue should be solved