Reply
New Contributor
Posts: 1
Registered: ‎04-08-2014

Hbase Custom Splitters in MapReduce

Hi,

 

Does anybody have an example on how to implement a custom splitter on MapReduce? By default, only one mapper is being spawned

per region.

I want to increase the number of mappers per region. Im using TableMapReduceUtil.initTableMapperJob in my MR job.

 

thanks.

 

 

 

Highlighted
Posts: 1,885
Kudos: 422
Solutions: 298
Registered: ‎07-31-2013

Re: Hbase Custom Splitters in MapReduce

Take a look at how the deprecated HBase's TableInputFormat (mapred API) created on-demand splits: https://github.com/cloudera/hbase/blob/cdh5.1.0-release/hbase-server/src/main/java/org/apache/hadoop...

You can basically apply the same logic within the mapreduce API TableInputFormat#getSplits(…) by inheriting and overriding it.
Announcements