Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

SolrResourceLoader cannot load files from outside the instance's directory

avatar
Rising Star

Hi,

 

I am trying to index hbase table with solr using batch indexer as per http://www.cloudera.com/documentation/archive/search/1-3-0/Cloudera-Search-User-Guide/csug_hbase_bat...

 

However, running 

 

hadoop --config /etc/hadoop/conf jar \
/usr/lib/hbase-solr/tools/hbase-indexer-mr-*-job.jar --conf \
/etc/hbase/conf/hbase-site.xml -D 'mapred.child.java.opts=-Xmx500m' \
--hbase-indexer-file $HOME/morphline-hbase-mapper.xml --zk-host \
127.0.0.1:2181/solr --collection hbase-collection1 --go-live --log4j \
src/test/resources/log4j.properties

yields the following exception:

 

 

Caused by: java.io.IOException: For security reasons, SolrResourceLoader cannot load files from outside the instance's directory: /var/lib/solr/solr_configs/solr_hbase_conf/solrconfig.xml; if you want to override this safety feature and you are sure about the consequences, you can pass the system property -Dsolr.allow.unsafe.resourceloading=true to your JVM

The instance directory is /var/lib/solr/hbase-core. 

 

Not entirely sure what file resources are in question here.

Could you please advice me on this matter? 

 

Thank you,

Gin

1 ACCEPTED SOLUTION

avatar
Rising Star

Lucky me! This works. Credit for the idea goes to: https://github.com/NGDATA/hbase-indexer/wiki/Batch-indexing

 

hadoop jar /opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hbase-solr/tools/hbase-indexer-mr-1.5-cdh5.4.11-job.jar  \
    --hbase-indexer-zk 127.0.0.1 \
    --hbase-indexer-name hbaseIndexer \
    --reducers 0

I guess all the info is picked up from the indexer.

View solution in original post

2 REPLIES 2

avatar
Rising Star

Please find more traces attached. It seems that the command does not pick up solr configuration from zookeeper and looks for it locally.

 

Caused by: org.apache.solr.common.SolrException: Error loading solr config from file:/tmp/hadoop-solr/mapred/local/1477069555232/dae8337e-e74b-4212-b513-3c4b000dfe30.solr.zip/file:/tmp/hadoop-solr/mapred/local/1477069555232/dae8337e-e74b-4212-b513-3c4b000dfe30.solr.zip/conf/solrconfig.xml
	at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:154)
	at org.apache.solr.core.ConfigSetService.createSolrConfig(ConfigSetService.java:80)
	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:61)
	... 14 more
Caused by: java.io.IOException: For security reasons, SolrResourceLoader cannot load files from outside the instance's directory: /var/lib/solr/solr_configs/solr_hbase_conf/solrconfig.xml; if you want to override this safety feature and you are sure about the consequences, you can pass the system property -Dsolr.allow.unsafe.resourceloading=true to your JVM
	at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:340)
	at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:308)
	at org.apache.solr.core.Config.<init>(Config.java:117)
	at org.apache.solr.core.Config.<init>(Config.java:87)
	at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:167)
	at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:145)
	... 16 more
16/10/21 17:06:00 ERROR hadoop.ForkedMapReduceIndexerTool: Job failed! jobName: HBaseMapReduceIndexerTool/HBaseIndexerMapper, jobId: job_local2112254315_0001
16/10/21 17:06:00 INFO hadoop.ForkedMapReduceIndexerTool: Deleting generated output directory /tmp/search-bbaef17e-c4d9-4e0c-adb1-39c830d53c3a

avatar
Rising Star

Lucky me! This works. Credit for the idea goes to: https://github.com/NGDATA/hbase-indexer/wiki/Batch-indexing

 

hadoop jar /opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hbase-solr/tools/hbase-indexer-mr-1.5-cdh5.4.11-job.jar  \
    --hbase-indexer-zk 127.0.0.1 \
    --hbase-indexer-name hbaseIndexer \
    --reducers 0

I guess all the info is picked up from the indexer.