Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Navigator Meta server not coming up

Navigator Meta server not coming up

Expert Contributor

After restarting CMS , navigator metadata server is not comign up and i see these errors in the logs.

 

Anyone has any pointers on how to get past this issue ? is this a bug ?

 

Oct 25, 9:18:04.061 AM WARN org.apache.solr.schema.IndexSchema
[coreLoadExecutor-5-thread-1]: Field user is not multivalued and destination for multiple copyFields (2)
Oct 25, 9:18:04.567 AM ERROR org.apache.solr.core.CoreContainer
[coreLoadExecutor-5-thread-1]: Error creating core [nav_elements]: Error opening new searcher
org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:881)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:654)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1574)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1686)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:853)
... 8 more
Caused by: java.io.EOFException: read past EOF: MMapIndexInput(path="/dcf/hdp/cloudera-scm-navigator/solr/nav_elements/data/index/_bjz6.fnm")
at org.apache.lucene.store.ByteBufferIndexInput.readByte(ByteBufferIndexInput.java:81)
at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:41)
at org.apache.lucene.store.DataInput.readInt(DataInput.java:98)
at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:134)
at org.apache.lucene.codecs.lucene46.Lucene46FieldInfosReader.read(Lucene46FieldInfosReader.java:57)
at org.apache.lucene.index.SegmentReader.readFieldInfos(SegmentReader.java:289)
at org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:864)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:816)
at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:279)
at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:111)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1537)
... 10 more

 

4 REPLIES 4

Re: Navigator Meta server not coming up

New Contributor

I'm having the same issue, has anyone found a fix for this yet?

Thanks

Chris

Re: Navigator Meta server not coming up

Expert Contributor

What i did was i backed up this directory and restarted the metadataserver  as a workaround. it wil rebuild all indexes from scratch. Just verify your audits after restart. 

 

sudo mv /dcf/hdp/cloudera-scm-navigator /dcf/hdp/cloudera-scm-navigator.bkp 

 

Restart Navigator 

Re: Navigator Meta server not coming up

New Contributor

Hi Desind,

 

Thanks for the quick reply and solution, my paths are different to the ones you describe so i did the following:

 

Stop Navigator

 

mv /var/lib/cloudera-scm-navigator /var/lib/cloudera-scm-navigator.bak

 

Start Navigator

 

The /var/lib/cloudera-scm-navigator was regnerated when Navigator was started but the following folfer was not regenerated: /var/lib/cloudera-scm-navigator/diagnosticData

 

The Navigator Metadata Server is working ok now but just wondered why this diagnosticData folder is missing.

 

Many Thanks,

Chris

Re: Navigator Meta server not coming up

Cloudera Employee

Hi

 

noticing this error 

"Caused by: java.io.EOFException: read past EOF: MMapIndexInput(path="/var/lib/you_metadata_path")"

 

When errors like this appear in the logs its usually a sign that the internal solr core index has become corrupted somehow from missing files, and the index will need to be regenerated again over time as fresh metadata comes in.

 

Stopping the Navigator Metadat Server, Clearing the metadata directory and starting a fresh index like this is the best option if this occurs.. It seems to occur most commonly from disks filling up, or a hard server crash of some sort.

 

-T

 


Tim Brigley, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Don't have an account?
Coming from Hortonworks? Activate your account here