Member since
10-13-2016
31
Posts
4
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2718 | 02-05-2018 03:12 AM | |
4882 | 12-09-2016 08:36 AM | |
1810 | 11-25-2016 07:14 AM | |
3285 | 11-09-2016 09:59 AM | |
3923 | 10-21-2016 10:36 AM |
02-05-2018
03:12 AM
Ok, my bad.
A quotation from stackoverflow:
-libjars makes Jars only available for JVMs running remote map and reduce task
To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable:
Therefore:
export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar
I'm using hbase-indexer to get solr dependencies.
Gin
... View more
02-02-2018
02:58 PM
Hi,
I am unable to add extra dependencies to the hbase-indexer batch job.
Neither HBASE_INDEXER_CLASSPATH nor --libjars seem to add anything to the classpath:
hadoop jar hbase-indexer-mr-job.jar \
--libjars dependencies.jar \
--hbase-indexer-zk 10.0.1.252:2181 \
--hbase-indexer-name collection \
--reducers 0
Env variable HBASE_INDEXER_CLASSPATH works as expected with the hbase-indexer process though.
Am I doing something wrong?
Thanks,
Gin
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Solr
-
MapReduce
07-14-2017
02:32 PM
Hi, Is it possible to store hbase-indexer's configuration, namely morphlines.conf, in Zookeeper instead of a local storage? Just like SolrCloud. Thanks, Gintas
... View more
Labels:
- Labels:
-
Apache Solr
03-14-2017
04:49 AM
Spot on, I missed the directory structure and just dumped my jars under plugins.d. The cmd line did not give me clues since it did not have anything related to plugin directories, e.g. "--plugins-path /usr/lib/flume-ng/plugins.d:/var/lib/flume-ng/plugins.d". Thanks!
... View more
03-13-2017
05:46 AM
Hi, A jar with custom flume components under default plugin folders /usr/lib/flume-ng/plugins.d or /var/lib/flume-ng/plugins.d is not picked up by Flume. I don't event see these folders included in the classpath. All works just fine if I add my jar to one of the default CP folders, e.g. /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/flume-ng/lib. Did anyone have a similar issue? Cheers, Gin
... View more
Labels:
- Labels:
-
Apache Flume
12-14-2016
02:18 PM
2 Kudos
Thanks for clarifying, Eric. Although I still think that a separate public/private cloudera-proprietary maven repo would be much more convenient. In the end, these JDBCs end up in (private) maven repos anyway - just takes more hassle. Best, Gin
... View more
12-13-2016
08:24 AM
Hi, Is there a particular reason why Impala JDBC driver is not in Cloudera's Maven Repo? Strange, because other CDH-related libraries are there. Best, Gin
... View more
Labels:
- Labels:
-
Apache Impala
12-09-2016
08:36 AM
Yes, the 0b-sized old WALs were the culprit. The hbase-indexer could not get past the corrupt files no matter what. Removing them was not an option as the hbase-indexer was expecting for these files. I did a dirty trick and copy-pasted the contents from other old WALs.The source WALs, judging from the size and contents, looked like empty carriers. That did the trick and hbase-indexer finally consumed all of the old WALs.
... View more
11-28-2016
07:52 AM
More info. I see in the oldWALs folder the oldest wal file size is 0. Can it be that this is a corrupt file and the indexer cannot get past it to process the rest of WAL records?
... View more