Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

SolrResourceLoader cannot load files from outside the instance's directory

Solved Go to solution

SolrResourceLoader cannot load files from outside the instance's directory

Contributor

Hi,

 

I am trying to index hbase table with solr using batch indexer as per http://www.cloudera.com/documentation/archive/search/1-3-0/Cloudera-Search-User-Guide/csug_hbase_bat...

 

However, running 

 

hadoop --config /etc/hadoop/conf jar \
/usr/lib/hbase-solr/tools/hbase-indexer-mr-*-job.jar --conf \
/etc/hbase/conf/hbase-site.xml -D 'mapred.child.java.opts=-Xmx500m' \
--hbase-indexer-file $HOME/morphline-hbase-mapper.xml --zk-host \
127.0.0.1:2181/solr --collection hbase-collection1 --go-live --log4j \
src/test/resources/log4j.properties

yields the following exception:

 

 

Caused by: java.io.IOException: For security reasons, SolrResourceLoader cannot load files from outside the instance's directory: /var/lib/solr/solr_configs/solr_hbase_conf/solrconfig.xml; if you want to override this safety feature and you are sure about the consequences, you can pass the system property -Dsolr.allow.unsafe.resourceloading=true to your JVM

The instance directory is /var/lib/solr/hbase-core. 

 

Not entirely sure what file resources are in question here.

Could you please advice me on this matter? 

 

Thank you,

Gin

1 ACCEPTED SOLUTION

Accepted Solutions

Re: SolrResourceLoader cannot load files from outside the instance's directory

Contributor

Lucky me! This works. Credit for the idea goes to: https://github.com/NGDATA/hbase-indexer/wiki/Batch-indexing

 

hadoop jar /opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hbase-solr/tools/hbase-indexer-mr-1.5-cdh5.4.11-job.jar  \
    --hbase-indexer-zk 127.0.0.1 \
    --hbase-indexer-name hbaseIndexer \
    --reducers 0

I guess all the info is picked up from the indexer.

2 REPLIES 2

Re: SolrResourceLoader cannot load files from outside the instance's directory

Contributor

Please find more traces attached. It seems that the command does not pick up solr configuration from zookeeper and looks for it locally.

 

Caused by: org.apache.solr.common.SolrException: Error loading solr config from file:/tmp/hadoop-solr/mapred/local/1477069555232/dae8337e-e74b-4212-b513-3c4b000dfe30.solr.zip/file:/tmp/hadoop-solr/mapred/local/1477069555232/dae8337e-e74b-4212-b513-3c4b000dfe30.solr.zip/conf/solrconfig.xml
	at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:154)
	at org.apache.solr.core.ConfigSetService.createSolrConfig(ConfigSetService.java:80)
	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:61)
	... 14 more
Caused by: java.io.IOException: For security reasons, SolrResourceLoader cannot load files from outside the instance's directory: /var/lib/solr/solr_configs/solr_hbase_conf/solrconfig.xml; if you want to override this safety feature and you are sure about the consequences, you can pass the system property -Dsolr.allow.unsafe.resourceloading=true to your JVM
	at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:340)
	at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:308)
	at org.apache.solr.core.Config.<init>(Config.java:117)
	at org.apache.solr.core.Config.<init>(Config.java:87)
	at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:167)
	at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:145)
	... 16 more
16/10/21 17:06:00 ERROR hadoop.ForkedMapReduceIndexerTool: Job failed! jobName: HBaseMapReduceIndexerTool/HBaseIndexerMapper, jobId: job_local2112254315_0001
16/10/21 17:06:00 INFO hadoop.ForkedMapReduceIndexerTool: Deleting generated output directory /tmp/search-bbaef17e-c4d9-4e0c-adb1-39c830d53c3a

Re: SolrResourceLoader cannot load files from outside the instance's directory

Contributor

Lucky me! This works. Credit for the idea goes to: https://github.com/NGDATA/hbase-indexer/wiki/Batch-indexing

 

hadoop jar /opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hbase-solr/tools/hbase-indexer-mr-1.5-cdh5.4.11-job.jar  \
    --hbase-indexer-zk 127.0.0.1 \
    --hbase-indexer-name hbaseIndexer \
    --reducers 0

I guess all the info is picked up from the indexer.