02-02-2018 02:58 PM - last edited on 02-03-2018 06:23 AM by cjervis
I am unable to add extra dependencies to the hbase-indexer batch job.
Neither HBASE_INDEXER_CLASSPATH nor --libjars seem to add anything to the classpath:
hadoop jar hbase-indexer-mr-job.jar \ --libjars dependencies.jar \ --hbase-indexer-zk 10.0.1.252:2181 \ --hbase-indexer-name collection \ --reducers 0
Env variable HBASE_INDEXER_CLASSPATH works as expected with the hbase-indexer process though.
Am I doing something wrong?
Solved! Go to Solution.
02-05-2018 03:12 AM - last edited on 02-05-2018 06:56 AM by cjervis
Ok, my bad.
A quotation from stackoverflow:
-libjars makes Jars only available for JVMs running remote map and reduce task
To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable:
export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar
I'm using hbase-indexer to get solr dependencies.