storm persistent storage to HBase in secure cluster

[#1]

Hi,

I had written a custom bolt extending, BaseBasicBolt NOT RichBolt, where i wrote my custom code to connect to HBase and write input stream into HBase table.

To do this, i am creating connections to HBase in my prepare method, as below and closing connections in cleanup().

public void prepare(Map stormConf, TopologyContext context)  {

    try {
        fmt = DateTimeFormat.forPattern("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");

        config = HBaseConfiguration.create();

        config.set("hadoop.security.authentication", "Kerberos");
        config.set("hbase.security.authentication", "kerberos");

        config.addResource("hdfs-site.xml");
        config.addResource("core-site.xml");
        config.addResource("hbase-site.xml");

        UserGroupInformation.setConfiguration(config);        
	UserGroupInformation.loginUserFromKeytab(KEYTAB_PRINC, KEYTAB_PATH);

        LOG.info("HBase Cnxn Done.");

        connection = ConnectionFactory.createConnection(config);
        table = connection.getTable(TableName.valueOf(HBASE_TABLE_NAME));

    }catch (Exception e)    {
        System.out.println("Exception occured during hbase connection preparations" + e.getMessage());
        e.printStackTrace();
    }

}

The issue is, when i am running my storm topology with 1executor and 1task, topology is running fine without any issues, but when i increase the executors/tasks, say 15executors & 60tasks, bolt where i am doing hbase connections is failing with authentication issues, like GSS exceptions!!!.

Can some one pls share your knowledge/insight into what gng wrong with my code. Thanks a lot...

[#2 · Accepted solution]

Maybe you're running into https://issues.apache.org/jira/browse/STORM-1521 ?

[#3]

Can you please me the git link for what changes have been done in that issue, to fix it. I could n't locate the exact chagnes from the link you shared. @Josh Elser. Thanks a lot.

[#4]

Thanks a lot @Josh Elser. That solved my issue. Thanks very much

[#5]

After following the link https://issues.apache.org/jira/browse/STORM-1521 i was able to fix the authentication problem i observed!!