Reply
Contributor
Posts: 31
Registered: ‎10-13-2016
Accepted Solution

SolrResourceLoader cannot load files from outside the instance's directory

Hi,

 

I am trying to index hbase table with solr using batch indexer as per http://www.cloudera.com/documentation/archive/search/1-3-0/Cloudera-Search-User-Guide/csug_hbase_bat...

 

However, running 

 

hadoop --config /etc/hadoop/conf jar \
/usr/lib/hbase-solr/tools/hbase-indexer-mr-*-job.jar --conf \
/etc/hbase/conf/hbase-site.xml -D 'mapred.child.java.opts=-Xmx500m' \
--hbase-indexer-file $HOME/morphline-hbase-mapper.xml --zk-host \
127.0.0.1:2181/solr --collection hbase-collection1 --go-live --log4j \
src/test/resources/log4j.properties

yields the following exception:

 

 

Caused by: java.io.IOException: For security reasons, SolrResourceLoader cannot load files from outside the instance's directory: /var/lib/solr/solr_configs/solr_hbase_conf/solrconfig.xml; if you want to override this safety feature and you are sure about the consequences, you can pass the system property -Dsolr.allow.unsafe.resourceloading=true to your JVM

The instance directory is /var/lib/solr/hbase-core. 

 

Not entirely sure what file resources are in question here.

Could you please advice me on this matter? 

 

Thank you,

Gin

Contributor
Posts: 31
Registered: ‎10-13-2016

Re: SolrResourceLoader cannot load files from outside the instance's directory

Please find more traces attached. It seems that the command does not pick up solr configuration from zookeeper and looks for it locally.

 

Caused by: org.apache.solr.common.SolrException: Error loading solr config from file:/tmp/hadoop-solr/mapred/local/1477069555232/dae8337e-e74b-4212-b513-3c4b000dfe30.solr.zip/file:/tmp/hadoop-solr/mapred/local/1477069555232/dae8337e-e74b-4212-b513-3c4b000dfe30.solr.zip/conf/solrconfig.xml
	at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:154)
	at org.apache.solr.core.ConfigSetService.createSolrConfig(ConfigSetService.java:80)
	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:61)
	... 14 more
Caused by: java.io.IOException: For security reasons, SolrResourceLoader cannot load files from outside the instance's directory: /var/lib/solr/solr_configs/solr_hbase_conf/solrconfig.xml; if you want to override this safety feature and you are sure about the consequences, you can pass the system property -Dsolr.allow.unsafe.resourceloading=true to your JVM
	at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:340)
	at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:308)
	at org.apache.solr.core.Config.<init>(Config.java:117)
	at org.apache.solr.core.Config.<init>(Config.java:87)
	at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:167)
	at org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:145)
	... 16 more
16/10/21 17:06:00 ERROR hadoop.ForkedMapReduceIndexerTool: Job failed! jobName: HBaseMapReduceIndexerTool/HBaseIndexerMapper, jobId: job_local2112254315_0001
16/10/21 17:06:00 INFO hadoop.ForkedMapReduceIndexerTool: Deleting generated output directory /tmp/search-bbaef17e-c4d9-4e0c-adb1-39c830d53c3a
Contributor
Posts: 31
Registered: ‎10-13-2016

Re: SolrResourceLoader cannot load files from outside the instance's directory

Lucky me! This works. Credit for the idea goes to: https://github.com/NGDATA/hbase-indexer/wiki/Batch-indexing

 

hadoop jar /opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hbase-solr/tools/hbase-indexer-mr-1.5-cdh5.4.11-job.jar  \
    --hbase-indexer-zk 127.0.0.1 \
    --hbase-indexer-name hbaseIndexer \
    --reducers 0

I guess all the info is picked up from the indexer.