Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hdfs fsck /solr failed for some collections

Highlighted

hdfs fsck /solr failed for some collections

Explorer

Hi!

 

After upgrade from 5.8.2 to 5.11 fsck generates an error for some SOLR collections (before upgrade no error was generated).

 

I see only one difference between other collections. FSCK fails on collection having no documents.

 

HADOOP_USER_NAME=hdfs hdfs fsck /solr/cases_ee/core_node1 -files
Connecting to namenode via http://catnn002.intrum.net:50070
FSCK started by hdfs (auth:SIMPLE) from /10.3.110.11 for path /solr/cases_ee/core_node1 at Mon Apr 24 22:24:57 CEST 2017
/solr/cases_ee/core_node1 <dir>
/solr/cases_ee/core_node1/data <dir>
/solr/cases_ee/core_node1/data/index <dir>
/solr/cases_ee/core_node1/data/index/HdfsDirectory@aba1bc58 lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@135f5ba6-write.lock 0 bytes, 0 block(s):  OK
/solr/cases_ee/core_node1/data/index/segments_1 53 bytes, 1 block(s):  OK
/solr/cases_ee/core_node1/data/snapshot_metadata <dir>
/solr/cases_ee/core_node1/data/tlog <dir>
FSCK ended at Mon Apr 24 22:24:57 CEST 2017 in 3 milliseconds
null


Fsck on path '/solr/cases_ee/core_node1' FAILED
HADOOP_USER_NAME=hdfs hadoop fs -ls -R /solr/cases_ee/core_node1

drwxr-xr-x   - solr solr          0 2017-04-24 21:55 /solr/cases_ee/core_node1/data
drwxr-xr-x   - solr solr          0 2017-04-24 21:55 /solr/cases_ee/core_node1/data/index
-rw-r--r--   3 solr solr          0 2017-04-24 21:55 /solr/cases_ee/core_node1/data/index/HdfsDirectory@aba1bc58 lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@135f5ba6-write.lock
-rwxr-xr-x   3 solr solr         53 2017-04-07 09:04 /solr/cases_ee/core_node1/data/index/segments_1
drwxr-xr-x   - solr solr          0 2017-04-24 21:55 /solr/cases_ee/core_node1/data/snapshot_metadata
drwxr-xr-x   - solr solr          0 2017-04-12 16:31 /solr/cases_ee/core_node1/data/tlog
-rw-r--r--   3 solr solr         81 2017-04-24 21:56 /solr/cases_ee/core_node1/data/tlog/tlog.0000000000000000008
-rw-r--r--   3 solr solr         81 2017-04-24 21:56 /solr/cases_ee/core_node1/data/tlog/tlog.0000000000000000010
2 REPLIES 2

Re: hdfs fsck /solr failed for some collections

Rising Star

Looks similar to another reported case:

 

http://community.cloudera.com/t5/Storage-Random-Access-HDFS/Cannot-get-a-file-on-HDFS-becouse-of-quo... and

https://issues.apache.org/jira/browse/HDFS-10788

 

Can you set log level to debug, rerun fsck and send NameNode log to me?

Re: hdfs fsck /solr failed for some collections

Explorer

Our night job had regenerated the SOLR index and fsck error gone. Sorry, I can not reproduce the error.

Don't have an account?
Coming from Hortonworks? Activate your account here