Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Adding a jar to hbase-indexer mapreduce job

Solved Go to solution
Highlighted

Adding a jar to hbase-indexer mapreduce job

Contributor

 

Hi,

 

I am unable to add extra dependencies to the hbase-indexer batch job.

Neither HBASE_INDEXER_CLASSPATH nor --libjars seem to add anything to the classpath:

 

hadoop jar hbase-indexer-mr-job.jar \
--libjars dependencies.jar \
--hbase-indexer-zk 10.0.1.252:2181 \
--hbase-indexer-name collection \
--reducers 0

 

Env variable HBASE_INDEXER_CLASSPATH works as expected with the hbase-indexer process though.

 

Am I doing something wrong? 

 

Thanks,

Gin

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Adding a jar to hbase-indexer mapreduce job

Contributor

Ok, my bad.

 

A quotation from stackoverflow:

  • -libjars makes Jars only available for JVMs running remote map and reduce task

  • To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable:

Therefore:

 

export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar

 I'm using hbase-indexer to get solr dependencies.

 

Gin

1 REPLY 1

Re: Adding a jar to hbase-indexer mapreduce job

Contributor

Ok, my bad.

 

A quotation from stackoverflow:

  • -libjars makes Jars only available for JVMs running remote map and reduce task

  • To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable:

Therefore:

 

export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar

 I'm using hbase-indexer to get solr dependencies.

 

Gin