We have kerberos enabled in the our dev cluster.. Hadoop has no issues reading/writing to Hbase but Spark on YARN throws all kinds of expection when it tries to read/write from hbase.

15/08/11 13:07:47 WARN security.UserGroupInformation: PriviledgedActionException as:usera (auth:SIMPLE) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

15/08/11 13:07:47 WARN ipc.AbstractRpcClient: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

15/08/11 13:07:47 ERROR ipc.AbstractRpcClient: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.

javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)

at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)

at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:605)

at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)

at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:731)

at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:728)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

This is our Hbase code:

  // Instantiating Configuration class

        Configuration config = HBaseConfiguration.create();

        System.setProperty("java.security.krb5.conf", "/etc/krb5.conf");

        System.setProperty("java.security.auth.login.config", "/etc/zookeeper/conf/jaas.conf");

        config.set("hbase.zookeeper.quorum", "host-node2");

        config.set("hbase.zookeeper.property.clientPort", "2181");

        UserGroupInformation.setConfiguration(config);

        UserGroupInformation.loginUserFromKeytab("usera@DOMAIN.COM", "/home/ubuntu/usera.keytab");

        // Instantiating HTable class

        HTable hTable = new HTable(config, "test");

        Put p = new Put(Bytes.toBytes("row100"));

        // adding values using add() method

        // accepts column family name, qualifier/row name ,value

        p.add(Bytes.toBytes("cf1"),

        Bytes.toBytes(columnQualifier),Bytes.toBytes(s));

        System.out.println("Added row to Hbase table");

        // Saving the put Instance to the HTable.

        hTable.put(p);

        hTable.close();

I've given permissions for usera to read/write from test table

What seem to be causing this? Any help much appreciated.