Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Storm-HBase: HbaseBolt does not write into the table

avatar
New Member

Hello,

I created the following topology:

Config config = new Config(); 
config.setClasspath("/usr/hdp/2.5.0.0-1245/hbase/lib");
config.setDebug(true); 
config.put(Config.TOPOLOGY_MAX_SPOUT_PENDING, 1); 
		
Map<String, Object> HBConf = new HashMap<String,Object>();
HBConf.put("hbase.rootdir","hdfs://localhost:8020/apps/hbase/data");
HBConf.put("zookeeper.znode.parent", "/hbase");

config.put("HBCONFIG",HBConf);
//TEST HBASE
SimpleHBaseMapper mapper = new SimpleHBaseMapper() 
        .withRowKeyField("row")
        .withColumnFields(new Fields("driverName"))
        .withColumnFamily("events");		

TopologyBuilder builder = new TopologyBuilder(); 
builder.setSpout("word-spout", new WordGenerator());
builder.setBolt("pre-hive", new PrepareTuple()).shuffleGrouping("word-spout");
builder.setBolt("hbase-bolt", new HBaseBolt("driver_dangerous_event", mapper).withConfigKey("HBCONFIG")).shuffleGrouping("pre-hive");
		
LocalCluster cluster = new LocalCluster(); 
cluster.submitTopology("HelloStorm", config, builder.createTopology()); 

This topology is built with three components:

WordGenerator - Just generates random word),

PrepareTuple - Prepare data to be inserted in HBase. It generates tree fields used in SimpleHBaseMapper class

public void execute(Tuple input) {
	this.collector.emit(new Values(id,input.getString(0), "events" ));
	id=id+1;
System.out.println("###PRE-HBASE### Emitting tuple:"+ Integer.toString(id) +","+ user.getNome());
	}
	
public void declareOutputFields(OutputFieldsDeclarer declarer) {
		declarer.declare(new Fields("row","driverName","events"));
}

The idea is to create a tuple to be written in the "driver_dangerous_event" HBASE table. Unfortunately, no rows are inserted in the table. This is the log of my topology but I can't see errors:

###PRE-HBASE### Emitting tuple:2,Word 1
7767 [Thread-16-pre-hive-executor[3 3]] INFO  o.a.s.d.executor - Execute done TUPLE source: word-spout:4, stream: default, id: {}, [Word 1] TASK: 3 DELTA: 
7776 [Thread-18-__acker-executor[1 1]] INFO  o.a.s.d.executor - Preparing bolt __acker:(1)
7787 [Thread-20-__system-executor[-1 -1]] INFO  o.a.s.d.executor - Preparing bolt __system:(-1)
7797 [Thread-18-__acker-executor[1 1]] INFO  o.a.s.d.executor - Prepared bolt __acker:(1)
7800 [Thread-20-__system-executor[-1 -1]] INFO  o.a.s.d.executor - Prepared bolt __system:(-1)
8341 [Thread-14-hbase-bolt-executor[2 2]] WARN  o.a.h.u.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
8752 [Thread-14-hbase-bolt-executor[2 2]] INFO  o.a.h.h.z.RecoverableZooKeeper - Process identifier=hconnection-0x2ea63a1 connecting to ZooKeeper ensemble=localhost:2181
9193 [Thread-14-hbase-bolt-executor[2 2]] INFO  o.a.s.d.executor - Prepared bolt hbase-bolt:(2)
9193 [Thread-14-hbase-bolt-executor[2 2]] INFO  o.a.s.d.executor - Processing received message FOR 2 TUPLE: source: pre-hive:3, stream: default, id: {}, [2, Word 1, events]
9198 [Thread-14-hbase-bolt-executor[2 2]] INFO  o.a.s.d.executor - Execute done TUPLE source: pre-hive:3, stream: default, id: {}, [2, Word 1, events] TASK: 2 DELTA: 
9198 [Thread-14-hbase-bolt-executor[2 2]] INFO  o.a.s.d.executor - Processing received message FOR -2 TUPLE: source: __system:-1, stream: __tick, id: {}, [1]
12709 [Thread-22-word-spout-executor[4 4]] INFO  o.a.s.d.task - Emitting: word-spout default [Word 1]
12710 [Thread-22-word-spout-executor[4 4]] INFO  o.a.s.d.executor - TRANSFERING tuple [dest: 3 tuple: source: word-spout:4, stream: default, id: {}, [Word 1]]


Is There something wrong here?

Thanks

1 ACCEPTED SOLUTION

avatar
New Member

FIXED. I updated the configuration as following:

Map<String, Object> HBConf = new HashMap<String,Object>();

HBConf.put("hbase.zookeeper.quorum", "server1, server2, server3");
/*Find server1,server2,server3 in your configuration ambari>HBASE>advanced>hbase.zookeeper.quorum */ 

HBConf.put("hbase.zookeeper.property.clientPort", "2181");
HBConf.put("hbase.master.port", "16000");

/*Find zookeeper.znode.parent in your configuration ambari>HBASE>advanced>zookeeper.znode.parent */
HBConf.put("zookeeper.znode.parent", "/hbase-unsecure");
		
config.put("HBCONFIG",HBConf);

View solution in original post

2 REPLIES 2

avatar
Super Guru

@Giuseppe Mento

just to test could you please try after updating line

builder.setBolt("hbase-bolt", new HBaseBolt("driver_dangerous_event", mapper).withConfigKey("HBCONFIG")).shuffleGrouping("pre-hive");

with this line

builder.setBolt("hbase-bolt", new HBaseBolt("driver_dangerous_event", mapper).withConfigKey("HBCONFIG")).fieldsGrouping("pre-hive",new Fields("row"));

avatar
New Member

FIXED. I updated the configuration as following:

Map<String, Object> HBConf = new HashMap<String,Object>();

HBConf.put("hbase.zookeeper.quorum", "server1, server2, server3");
/*Find server1,server2,server3 in your configuration ambari>HBASE>advanced>hbase.zookeeper.quorum */ 

HBConf.put("hbase.zookeeper.property.clientPort", "2181");
HBConf.put("hbase.master.port", "16000");

/*Find zookeeper.znode.parent in your configuration ambari>HBASE>advanced>zookeeper.znode.parent */
HBConf.put("zookeeper.znode.parent", "/hbase-unsecure");
		
config.put("HBCONFIG",HBConf);