question Re: Adding a jar to hbase-indexer mapreduce job in Archives of Support Questions (Read Only)

Adding a jar to hbase-indexer mapreduce job

ginsul — Fri, 16 Sep 2022 12:49:09 GMT

Hi,

I am unable to add extra dependencies to the hbase-indexer batch job.

Neither HBASE_INDEXER_CLASSPATH nor --libjars seem to add anything to the classpath:

hadoop jar hbase-indexer-mr-job.jar \
--libjars dependencies.jar \
--hbase-indexer-zk 10.0.1.252:2181 \
--hbase-indexer-name collection \
--reducers 0

Env variable HBASE_INDEXER_CLASSPATH works as expected with the hbase-indexer process though.

Am I doing something wrong?

Thanks,

Gin

Re: Adding a jar to hbase-indexer mapreduce job

ginsul — Mon, 05 Feb 2018 14:56:16 GMT

Ok, my bad.

A quotation from stackoverflow:

-libjars makes Jars only available for JVMs running remote map and reduce task
To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable:

Therefore:

export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar

I'm using hbase-indexer to get solr dependencies.

Gin