Storm-HBase: HbaseBolt does not write into the table

giuseppe_mento — Mon, 19 Dec 2016 02:19:33 GMT

Hello,

I created the following topology:

Config config = new Config(); 
config.setClasspath("/usr/hdp/2.5.0.0-1245/hbase/lib");
config.setDebug(true); 
config.put(Config.TOPOLOGY_MAX_SPOUT_PENDING, 1); 
		
Map<String, Object> HBConf = new HashMap<String,Object>();
HBConf.put("hbase.rootdir","hdfs://localhost:8020/apps/hbase/data");
HBConf.put("zookeeper.znode.parent", "/hbase");

config.put("HBCONFIG",HBConf);
//TEST HBASE
SimpleHBaseMapper mapper = new SimpleHBaseMapper() 
        .withRowKeyField("row")
        .withColumnFields(new Fields("driverName"))
        .withColumnFamily("events");		

TopologyBuilder builder = new TopologyBuilder(); 
builder.setSpout("word-spout", new WordGenerator());
builder.setBolt("pre-hive", new PrepareTuple()).shuffleGrouping("word-spout");
builder.setBolt("hbase-bolt", new HBaseBolt("driver_dangerous_event", mapper).withConfigKey("HBCONFIG")).shuffleGrouping("pre-hive");
		
LocalCluster cluster = new LocalCluster(); 
cluster.submitTopology("HelloStorm", config, builder.createTopology());

This topology is built with three components:

WordGenerator - Just generates random word),

PrepareTuple - Prepare data to be inserted in HBase. It generates tree fields used in SimpleHBaseMapper class

public void execute(Tuple input) {
	this.collector.emit(new Values(id,input.getString(0), "events" ));
	id=id+1;
System.out.println("###PRE-HBASE### Emitting tuple:"+ Integer.toString(id) +","+ user.getNome());
	}
	
public void declareOutputFields(OutputFieldsDeclarer declarer) {
		declarer.declare(new Fields("row","driverName","events"));
}

The idea is to create a tuple to be written in the "driver_dangerous_event" HBASE table. Unfortunately, no rows are inserted in the table. This is the log of my topology but I can't see errors:

###PRE-HBASE### Emitting tuple:2,Word 1
7767 [Thread-16-pre-hive-executor[3 3]] INFO  o.a.s.d.executor - Execute done TUPLE source: word-spout:4, stream: default, id: {}, [Word 1] TASK: 3 DELTA: 
7776 [Thread-18-__acker-executor[1 1]] INFO  o.a.s.d.executor - Preparing bolt __acker:(1)
7787 [Thread-20-__system-executor[-1 -1]] INFO  o.a.s.d.executor - Preparing bolt __system:(-1)
7797 [Thread-18-__acker-executor[1 1]] INFO  o.a.s.d.executor - Prepared bolt __acker:(1)
7800 [Thread-20-__system-executor[-1 -1]] INFO  o.a.s.d.executor - Prepared bolt __system:(-1)
8341 [Thread-14-hbase-bolt-executor[2 2]] WARN  o.a.h.u.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
8752 [Thread-14-hbase-bolt-executor[2 2]] INFO  o.a.h.h.z.RecoverableZooKeeper - Process identifier=hconnection-0x2ea63a1 connecting to ZooKeeper ensemble=localhost:2181
9193 [Thread-14-hbase-bolt-executor[2 2]] INFO  o.a.s.d.executor - Prepared bolt hbase-bolt:(2)
9193 [Thread-14-hbase-bolt-executor[2 2]] INFO  o.a.s.d.executor - Processing received message FOR 2 TUPLE: source: pre-hive:3, stream: default, id: {}, [2, Word 1, events]
9198 [Thread-14-hbase-bolt-executor[2 2]] INFO  o.a.s.d.executor - Execute done TUPLE source: pre-hive:3, stream: default, id: {}, [2, Word 1, events] TASK: 2 DELTA: 
9198 [Thread-14-hbase-bolt-executor[2 2]] INFO  o.a.s.d.executor - Processing received message FOR -2 TUPLE: source: __system:-1, stream: __tick, id: {}, [1]
12709 [Thread-22-word-spout-executor[4 4]] INFO  o.a.s.d.task - Emitting: word-spout default [Word 1]
12710 [Thread-22-word-spout-executor[4 4]] INFO  o.a.s.d.executor - TRANSFERING tuple [dest: 3 tuple: source: word-spout:4, stream: default, id: {}, [Word 1]]

Is There something wrong here?

Thanks

Re: Storm-HBase: HbaseBolt does not write into the table

rajkumar_singh — Mon, 19 Dec 2016 02:37:26 GMT

@Giuseppe Mento

just to test could you please try after updating line

builder.setBolt("hbase-bolt", new HBaseBolt("driver_dangerous_event", mapper).withConfigKey("HBCONFIG")).shuffleGrouping("pre-hive");

with this line

builder.setBolt("hbase-bolt", new HBaseBolt("driver_dangerous_event", mapper).withConfigKey("HBCONFIG")).fieldsGrouping("pre-hive",new Fields("row"));

Re: Storm-HBase: HbaseBolt does not write into the table

giuseppe_mento — Mon, 19 Dec 2016 03:45:07 GMT

FIXED. I updated the configuration as following:

Map<String, Object> HBConf = new HashMap<String,Object>();

HBConf.put("hbase.zookeeper.quorum", "server1, server2, server3");
/*Find server1,server2,server3 in your configuration ambari>HBASE>advanced>hbase.zookeeper.quorum */ 

HBConf.put("hbase.zookeeper.property.clientPort", "2181");
HBConf.put("hbase.master.port", "16000");

/*Find zookeeper.znode.parent in your configuration ambari>HBASE>advanced>zookeeper.znode.parent */
HBConf.put("zookeeper.znode.parent", "/hbase-unsecure");
		
config.put("HBCONFIG",HBConf);

question Re: Storm-HBase: HbaseBolt does not write into the table in Archives of Support Questions (Read Only)

Storm-HBase: HbaseBolt does not write into the table

Re: Storm-HBase: HbaseBolt does not write into the table

Re: Storm-HBase: HbaseBolt does not write into the table