Created on 12-07-2017 05:33 AM - edited 09-16-2022 05:36 AM
this problem bother me long time, please help me!
hbase cluster with kerberos works well.
my java code for anthentication as below
    static {
        System.setProperty("java.security.krb5.conf",DeviceChannelHbaseClient.class.getResource("/krb5.conf").getPath());
        
        UserGroupInformation.setConfiguration(HBASE_CONF);
       
        try {
        	
        	   URL keyTabUrl = DeviceChannelHbaseClient.class.getResource("/hbase-rw.keytab");
         	  if (keyTabUrl != null) {
         		  UserGroupInformation.loginUserFromKeytab("hbase-rw/hz@HZ.NETEASE.COM", DeviceChannelHbaseClient.class.getResource("/hbase-rw.keytab").getPath());
         	  }
        } catch (IOException e) {
              e.printStackTrace();
        }
    }at the starting, everythings works, but after a few days, an exception accurs:
[WARN ]16:49:48,007, [Class]RpcClientImpl, Exception encountered while connecting to the server : java.lang.IllegalStateException: This ticket is
 no longer valid
[WARN ]16:49:48,007, [Class]UserGroupInformation, PriviledgedActionException as:hbase-rw/hz@HZ.CLOUD.COM (auth:KERBEROS) cause:java.io.IOExcept
ion: java.lang.IllegalStateException: This ticket is no longer valid
[WARN ]16:49:50,131, [Class]RpcClientImpl, Exception encountered while connecting to the server : java.lang.IllegalStateException: This ticket is
 no longer valid
[WARN ]16:49:50,131, [Class]UserGroupInformation, PriviledgedActionException as:hbase-rw/hz@HZ.CLOUD.COM (auth:KERBEROS) cause:java.io.IOExcept
ion: java.lang.IllegalStateException: This ticket is no longer valid
[WARN ]16:49:52,154, [Class]UserGroupInformation, PriviledgedActionException as:hbase-rw/hz@HZ.CLOUD.COM (auth:KERBEROS) cause:javax.security.s
asl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
[WARN ]16:49:52,154, [Class]RpcClientImpl, Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate
 failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
[FATAL]16:49:52,165, [Class]RpcClientImpl, SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find an
y Kerberos tgt)]
        at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
        at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:181)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:618)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:163)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:744)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:741)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:741)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:907)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:874)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1243)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:34070)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1594)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1398)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1199)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1166)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1150)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:971)
        at org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83)
        at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:79)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
        at org.apache.hadoop.hbase.client.HTable.get(HTable.java:930)
        at org.apache.hadoop.hbase.client.HTable.exists(HTable.java:1436)
        at com.netease.yx.hbase.LMRegistUserHbase.exists(LMRegistUserHbase.java:270)
        at com.netease.yx.leave.motivate.LMLogConsumer$TestConsumeRunner.dealWebLeaveLog(LMLogConsumer.java:132)
        at com.netease.yx.leave.motivate.LMLogConsumer$TestConsumeRunner.run(LMLogConsumer.java:268)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
        at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
        at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
        at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:34070)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1594)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1398)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1199)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1166)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1150)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:971)
        at org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83)
        at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:79)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
        at org.apache.hadoop.hbase.client.HTable.get(HTable.java:930)
        at org.apache.hadoop.hbase.client.HTable.exists(HTable.java:1436)
        at com.netease.yx.hbase.LMRegistUserHbase.exists(LMRegistUserHbase.java:270)
        at com.netease.yx.leave.motivate.LMLogConsumer$TestConsumeRunner.dealWebLeaveLog(LMLogConsumer.java:132)
        at com.netease.yx.leave.motivate.LMLogConsumer$TestConsumeRunner.run(LMLogConsumer.java:268)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
        at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
        at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
        at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
        at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
        ... 32 morewhy? please help me , thanks very much.
Created 12-08-2017 03:09 AM
Hello,
The ticket you acquire from the keytab has an expiry date and a max renewable date.
So, if you see that error after a few days, it might just be that (either the expiry date or the max renewable date).
You need to "handle" these cases.
regards,
Mathieu
Created 12-13-2017 12:05 AM
I think the keytab you used has expired.
Try to kinit a new keytab for your code, and issue should be solved
 
					
				
				
			
		
