About ginsul

ginsul · ‎06-14-2018

Yes

ginsul · ‎02-05-2018

Ok, my bad. A quotation from stackoverflow: -libjars makes Jars only available for JVMs running remote map and reduce task To make these same JAR’s available to the client JVM (The JVM that’s created when you run the hadoop jar command) need to set HADOOP_CLASSPATH environment variable: Therefore: export HADOOP_CLASSPATH=`hbase-indexer classpath`:dependencies.jar I'm using hbase-indexer to get solr dependencies. Gin

ginsul · ‎02-02-2018

Hi, I am unable to add extra dependencies to the hbase-indexer batch job. Neither HBASE_INDEXER_CLASSPATH nor --libjars seem to add anything to the classpath: hadoop jar hbase-indexer-mr-job.jar \ --libjars dependencies.jar \ --hbase-indexer-zk 10.0.1.252:2181 \ --hbase-indexer-name collection \ --reducers 0 Env variable HBASE_INDEXER_CLASSPATH works as expected with the hbase-indexer process though. Am I doing something wrong? Thanks, Gin

ginsul · ‎07-14-2017

Hi, Is it possible to store hbase-indexer's configuration, namely morphlines.conf, in Zookeeper instead of a local storage? Just like SolrCloud. Thanks, Gintas

ginsul · ‎03-14-2017

Spot on, I missed the directory structure and just dumped my jars under plugins.d. The cmd line did not give me clues since it did not have anything related to plugin directories, e.g. "--plugins-path /usr/lib/flume-ng/plugins.d:/var/lib/flume-ng/plugins.d". Thanks!

ginsul · ‎03-13-2017

Hi, A jar with custom flume components under default plugin folders /usr/lib/flume-ng/plugins.d or /var/lib/flume-ng/plugins.d is not picked up by Flume. I don't event see these folders included in the classpath. All works just fine if I add my jar to one of the default CP folders, e.g. /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/flume-ng/lib. Did anyone have a similar issue? Cheers, Gin

ginsul · ‎12-14-2016

Thanks for clarifying, Eric. Although I still think that a separate public/private cloudera-proprietary maven repo would be much more convenient. In the end, these JDBCs end up in (private) maven repos anyway - just takes more hassle. Best, Gin

ginsul · ‎12-13-2016

Hi, Is there a particular reason why Impala JDBC driver is not in Cloudera's Maven Repo? Strange, because other CDH-related libraries are there. Best, Gin

ginsul · ‎12-09-2016

Yes, the 0b-sized old WALs were the culprit. The hbase-indexer could not get past the corrupt files no matter what. Removing them was not an option as the hbase-indexer was expecting for these files. I did a dirty trick and copy-pasted the contents from other old WALs.The source WALs, judging from the size and contents, looked like empty carriers. That did the trick and hbase-indexer finally consumed all of the old WALs.

ginsul · ‎11-28-2016

More info. I see in the oldWALs folder the oldest wal file size is 0. Can it be that this is a corrupt file and the indexer cannot get past it to process the rest of WAL records?

Online	Offline
Last Visited	‎10-19-2023 05:32 AM

Member Since	‎10-13-2016 08:56 AM
Last Visited	‎10-19-2023 05:32 AM
Posts	31
Kudos received	4

Cloudera Community

Re: Adding a jar to hbase-indexer mapreduce job

Re: RegionServer throws "not a SequenceFile", oldW...

Re: hbase-indexer and a collection with multiple c...

Re: Getting hbase rowkey in morphlines

Re: SolrResourceLoader cannot load files from outs...

Re: How do I add a library to hbase-indexer classp...

Re: Adding a jar to hbase-indexer mapreduce job

Adding a jar to hbase-indexer mapreduce job

hbase-indexer's configuration in Zookeeper?

Re: Flume plugin directories are not in classpath

Flume plugin directories are not in classpath

Re: Impala JDBC is not in maven repo

Impala JDBC is not in maven repo

Re: RegionServer throws "not a SequenceFile", oldW...

Re: RegionServer throws "not a SequenceFile", oldW...