question Re: SolrResourceLoader cannot load files from outside the instance's directory in Archives of Support Questions (Read Only)

SolrResourceLoader cannot load files from outside the instance's directory

ginsul — Fri, 16 Sep 2022 10:45:11 GMT

Hi,

I am trying to index hbase table with solr using batch indexer as per http://www.cloudera.com/documentation/archive/search/1-3-0/Cloudera-Search-User-Guide/csug_hbase_batch_indexer.html

However, running

hadoop --config /etc/hadoop/conf jar \
/usr/lib/hbase-solr/tools/hbase-indexer-mr-*-job.jar --conf \
/etc/hbase/conf/hbase-site.xml -D 'mapred.child.java.opts=-Xmx500m' \
--hbase-indexer-file $HOME/morphline-hbase-mapper.xml --zk-host \
127.0.0.1:2181/solr --collection hbase-collection1 --go-live --log4j \
src/test/resources/log4j.properties

yields the following exception:

Caused by: java.io.IOException: For security reasons, SolrResourceLoader cannot load files from outside the instance's directory: /var/lib/solr/solr_configs/solr_hbase_conf/solrconfig.xml; if you want to override this safety feature and you are sure about the consequences, you can pass the system property -Dsolr.allow.unsafe.resourceloading=true to your JVM

The instance directory is /var/lib/solr/hbase-core.

Not entirely sure what file resources are in question here.

Could you please advice me on this matter?

Thank you,

Gin

Re: SolrResourceLoader cannot load files from outside the instance's directory

ginsul — Fri, 21 Oct 2016 17:08:47 GMT

Please find more traces attached. It seems that the command does not pick up solr configuration from zookeeper and looks for it locally.

Caused by: org.apache.solr.common.SolrException: Error loading solr config from file:/tmp/hadoop-solr/mapred/local/1477069555232/dae8337e-e74b-4212-b513-3c4b000dfe30.solr.zip/file:/tmp/hadoop-solr/mapred/local/1477069555232/dae8337e-e74b-4212-b513-3c4b000dfe30.solr.zip/conf/solrconfig.xml
	at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:154)
	at org.apache.solr.core.ConfigSetService.createSolrConfig(ConfigSetService.java:80)
	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:61)
	... 14 more
Caused by: java.io.IOException: For security reasons, SolrResourceLoader cannot load files from outside the instance's directory: /var/lib/solr/solr_configs/solr_hbase_conf/solrconfig.xml; if you want to override this safety feature and you are sure about the consequences, you can pass the system property -Dsolr.allow.unsafe.resourceloading=true to your JVM
	at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:340)
	at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:308)
	at org.apache.solr.core.Config.<init>(Config.java:117)
	at org.apache.solr.core.Config.<init>(Config.java:87)
	at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:167)
	at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:145)
	... 16 more
16/10/21 17:06:00 ERROR hadoop.ForkedMapReduceIndexerTool: Job failed! jobName: HBaseMapReduceIndexerTool/HBaseIndexerMapper, jobId: job_local2112254315_0001
16/10/21 17:06:00 INFO hadoop.ForkedMapReduceIndexerTool: Deleting generated output directory /tmp/search-bbaef17e-c4d9-4e0c-adb1-39c830d53c3a

Re: SolrResourceLoader cannot load files from outside the instance's directory

ginsul — Fri, 21 Oct 2016 17:36:01 GMT

Lucky me! This works. Credit for the idea goes to: https://github.com/NGDATA/hbase-indexer/wiki/Batch-indexing

hadoop jar /opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hbase-solr/tools/hbase-indexer-mr-1.5-cdh5.4.11-job.jar  \
    --hbase-indexer-zk 127.0.0.1 \
    --hbase-indexer-name hbaseIndexer \
    --reducers 0

I guess all the info is picked up from the indexer.