Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Who Agreed with this topic

Spark jobs fail communicating with Hbase in a kerberos cluster

Explorer

We have kerberos enabled in the our dev cluster.. Hadoop has no issues reading/writing to Hbase but Spark on YARN throws all kinds of expection when it tries to read/write from hbase.

 

15/08/11 13:07:47 WARN security.UserGroupInformation: PriviledgedActionException as:usera (auth:SIMPLE) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

15/08/11 13:07:47 WARN ipc.AbstractRpcClient: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

15/08/11 13:07:47 ERROR ipc.AbstractRpcClient: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.

javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)

at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)

at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:605)

at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)

at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:731)

at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:728)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

 

 

This is our Hbase code:

 

 

  // Instantiating Configuration class

        Configuration config = HBaseConfiguration.create();

        

        System.setProperty("java.security.krb5.conf", "/etc/krb5.conf");

        System.setProperty("java.security.auth.login.config", "/etc/zookeeper/conf/jaas.conf");

        

        config.set("hbase.zookeeper.quorum", "host-node2");

        config.set("hbase.zookeeper.property.clientPort", "2181");

 

        UserGroupInformation.setConfiguration(config);

        

        UserGroupInformation.loginUserFromKeytab("usera@DOMAIN.COM", "/home/ubuntu/usera.keytab");

 

 

        // Instantiating HTable class

        HTable hTable = new HTable(config, "test");

        Put p = new Put(Bytes.toBytes("row100"));

 

        // adding values using add() method

        // accepts column family name, qualifier/row name ,value

                

        p.add(Bytes.toBytes("cf1"),

        Bytes.toBytes(columnQualifier),Bytes.toBytes(s));

 

        System.out.println("Added row to Hbase table");

        // Saving the put Instance to the HTable.

        hTable.put(p);

        hTable.close();

 

 

I've given permissions for usera to read/write from test table

 

 

What seem to be causing this? Any help much appreciated.

Who Agreed with this topic