Created on 02-02-2018 02:58 PM - edited 09-16-2022 05:49 AM
Hi,
I am unable to add extra dependencies to the hbase-indexer batch job.
Neither HBASE_INDEXER_CLASSPATH nor --libjars seem to add anything to the classpath:
hadoop jar hbase-indexer-mr-job.jar \ --libjars dependencies.jar \ --hbase-indexer-zk 10.0.1.252:2181 \ --hbase-indexer-name collection \ --reducers 0
Env variable HBASE_INDEXER_CLASSPATH works as expected with the hbase-indexer process though.
Am I doing something wrong?
Thanks,
Gin
Ok, my bad.
A quotation from stackoverflow:
-libjars makes Jars only available for JVMs running remote map and reduce task
To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable:
Therefore:
export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar
I'm using hbase-indexer to get solr dependencies.
Gin
Ok, my bad.
A quotation from stackoverflow:
-libjars makes Jars only available for JVMs running remote map and reduce task
To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable:
Therefore:
export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar
I'm using hbase-indexer to get solr dependencies.
Gin