Reply
Contributor
Posts: 28
Registered: ‎10-13-2016
Accepted Solution

Adding a jar to hbase-indexer mapreduce job

[ Edited ]

 

Hi,

 

I am unable to add extra dependencies to the hbase-indexer batch job.

Neither HBASE_INDEXER_CLASSPATH nor --libjars seem to add anything to the classpath:

 

hadoop jar hbase-indexer-mr-job.jar \
--libjars dependencies.jar \
--hbase-indexer-zk 10.0.1.252:2181 \
--hbase-indexer-name collection \
--reducers 0

 

Env variable HBASE_INDEXER_CLASSPATH works as expected with the hbase-indexer process though.

 

Am I doing something wrong? 

 

Thanks,

Gin

Contributor
Posts: 28
Registered: ‎10-13-2016

Re: Adding a jar to hbase-indexer mapreduce job

[ Edited ]

Ok, my bad.

 

A quotation from stackoverflow:

  • -libjars makes Jars only available for JVMs running remote map and reduce task

  • To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable:

Therefore:

 

export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar

 I'm using hbase-indexer to get solr dependencies.

 

Gin

Announcements