Support Questions
Find answers, ask questions, and share your expertise

Solr collection indexed to pdf( topic with Paragraph) in hdfs throws error during solr restart

New Contributor

When I restart Solr it throws the following error. Solr collection indexed to pdf in HDFS throws an error during Solr restart.I have created the collection in Solr which will index the pdf files and this collection is indexing all the pdf in HDFS.

java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: Unable to create core [PDFIndex]
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.apache.solr.core.CoreContainer.lambda$load$6(CoreContainer.java:594)
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Unable to create core [PDFIndex]
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:966)
	at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565)
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
	... 5 more
Caused by: org.apache.solr.common.SolrException: Index dir 'hdfs://192.168.1.16:8020/PDFIndex/data/index/' of core 'PDFIndex' is already locked. The most likely cause is another Solr server (or another solr core in this server) also configured to use this directory; other possible causes may be specific to lockType: hdfs
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:950)
	... 7 more
Caused by: org.apache.lucene.store.LockObtainFailedException: Index dir 'hdfs://192.168.1.16:8020/PDFIndex/data/index/' of core 'PDFIndex' is already locked. The most likely cause is another Solr server (or another solr core in this server) also configured to use this directory; other possible causes may be specific to lockType: hdfs
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:712)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:923)
	... 9 more
2018-12-22 07:55:13.431 ERROR (OldIndexDirectoryCleanupThreadForCore-PDFIndex) [   x:PDFIndex] o.a.s.c.HdfsDirectoryFactory Error checking for old index directories to clean-up.
java.io.IOException: Filesystem closed
	at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:808)
	at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2083)
	at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2069)
	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:791)
	at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
	at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
	at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1517)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1557)
	at org.apache.solr.core.HdfsDirectoryFactory.cleanupOldIndexDirectories(HdfsDirectoryFactory.java:546)
	at org.apache.solr.core.SolrCore.lambda$cleanupOldIndexDirectories$19(SolrCore.java:3050)
	at java.lang.Thread.run(Thread.java:748)
2018-12-22 07:55:13.433 ERROR (OldIndexDirectoryCleanupThreadForCore-PDFIndex) [   x:PDFIndex] o.a.s.c.SolrCore Failed to cleanup old index directories for core PDFIndex
java.lang.NullPointerException
	at org.apache.solr.core.HdfsDirectoryFactory.cleanupOldIndexDirectories(HdfsDirectoryFactory.java:564)
	at org.apache.solr.core.SolrCore.lambda$cleanupOldIndexDirectories$19(SolrCore.java:3050)
	at java.lang.Thread.run(Thread.java:748)
1 REPLY 1

Super Mentor

@Vaibhav Shukla

What is your SOLR version?

A very similar NullPointerException is reported here: https://issues.apache.org/jira/browse/SOLR-11381