Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Adding a jar to hbase-indexer mapreduce job

avatar
Rising Star

 

Hi,

 

I am unable to add extra dependencies to the hbase-indexer batch job.

Neither HBASE_INDEXER_CLASSPATH nor --libjars seem to add anything to the classpath:

 

hadoop jar hbase-indexer-mr-job.jar \
--libjars dependencies.jar \
--hbase-indexer-zk 10.0.1.252:2181 \
--hbase-indexer-name collection \
--reducers 0

 

Env variable HBASE_INDEXER_CLASSPATH works as expected with the hbase-indexer process though.

 

Am I doing something wrong? 

 

Thanks,

Gin

1 ACCEPTED SOLUTION

avatar
Rising Star

Ok, my bad.

 

A quotation from stackoverflow:

  • -libjars makes Jars only available for JVMs running remote map and reduce task

  • To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable:

Therefore:

 

export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar

 I'm using hbase-indexer to get solr dependencies.

 

Gin

View solution in original post

1 REPLY 1

avatar
Rising Star

Ok, my bad.

 

A quotation from stackoverflow:

  • -libjars makes Jars only available for JVMs running remote map and reduce task

  • To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable:

Therefore:

 

export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar

 I'm using hbase-indexer to get solr dependencies.

 

Gin