storm persistent storage to HBase in secure cluster

raja_aravapalli — Thu, 28 Apr 2016 05:09:18 GMT

Hi,

I had written a custom bolt extending, BaseBasicBolt NOT RichBolt, where i wrote my custom code to connect to HBase and write input stream into HBase table.

To do this, i am creating connections to HBase in my prepare method, as below and closing connections in cleanup().

public void prepare(Map stormConf, TopologyContext context)  {

    try {
        fmt = DateTimeFormat.forPattern("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");

        config = HBaseConfiguration.create();

        config.set("hadoop.security.authentication", "Kerberos");
        config.set("hbase.security.authentication", "kerberos");

        config.addResource("hdfs-site.xml");
        config.addResource("core-site.xml");
        config.addResource("hbase-site.xml");

        UserGroupInformation.setConfiguration(config);        
	UserGroupInformation.loginUserFromKeytab(KEYTAB_PRINC, KEYTAB_PATH);

        LOG.info("HBase Cnxn Done.");

        connection = ConnectionFactory.createConnection(config);
        table = connection.getTable(TableName.valueOf(HBASE_TABLE_NAME));

    }catch (Exception e)    {
        System.out.println("Exception occured during hbase connection preparations" + e.getMessage());
        e.printStackTrace();
    }

}

The issue is, when i am running my storm topology with 1executor and 1task, topology is running fine without any issues, but when i increase the executors/tasks, say 15executors & 60tasks, bolt where i am doing hbase connections is failing with authentication issues, like GSS exceptions!!!.

Can some one pls share your knowledge/insight into what gng wrong with my code. Thanks a lot...

Re: storm persistent storage to HBase in secure cluster

raja_aravapalli — Thu, 28 Apr 2016 08:28:50 GMT

After following the link https://issues.apache.org/jira/browse/STORM-1521 i was able to fix the authentication problem i observed!!

question Re: storm persistent storage to HBase in secure cluster in Archives of Support Questions (Read Only)

storm persistent storage to HBase in secure cluster

Re: storm persistent storage to HBase in secure cluster