Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Navigator metadata server keep on crashing after changing to Mysql DB

Solved Go to solution
Highlighted

Navigator metadata server keep on crashing after changing to Mysql DB

Contributor

Hi there,

 

I have enabled Navigator metadata server to use mysql db, but it keep on crashing with solr error, hope someone can help on this.

 

Thanks

 

SolrCore
[qtp2123908275-75]: org.apache.solr.common.SolrException: Cursor functionality requires a sort containing a uniqueKey field tie breaker
	at org.apache.solr.search.CursorMark.<init>(CursorMark.java:104)
	at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:179)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1476)
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:256)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
	at com.cloudera.nav.auth.SharedSecretRequestFilter.doFilter(SharedSecretRequestFilter.java:55)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:370)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:745)
1 ACCEPTED SOLUTION

Accepted Solutions

Re: Navigator metadata server keep on crashing after changing to Mysql DB

New Contributor

I had to reset the Navigator metadata server and followed the steps below:

1) Stop Navigator Metadata Server from CM

2) Take a back up of Navigator Metadata Server Storage Dir (if you want to archive the old data)

3) mv <storage_dir> < new_location>

4) Back up your navms database

5) In NMS database run:

"DELETE FROM NAV_UPGRADE_ORDINAL;" "INSERT INTO NAV_UPGRADE_ORDINAL VALUES (-1, -1);"

6) Start Navigator Metadata Server

 

this will delete all metadata, custom data added in the Navigator UI but does not affect the Audit logs visible under Navigator UI -> Audits.

6 REPLIES 6

Re: Navigator metadata server keep on crashing after changing to Mysql DB

New Contributor

Hi, can you find a solution for this issue?

Thanks

Re: Navigator metadata server keep on crashing after changing to Mysql DB

Super Guru

@elifsurmeli,

 

This usually indicates that the Solr data is out of sync (version-wise) with what nav_upgrade_ordinal table in your Navigator Metadata Server database.

 

Please do the following to see if that is the cause:

 

(1)  on the Navigator Metadata host, run rpm -qa |grep cloudera and post the results (so we can confirm the version of Navigator

 

(2)  Run the following query in your Navigator Metadata Server database (via your favorite db tool):

 

select * from NAV_UPGRADE_ORDINAL;

 

NOTE:  The above assumes that you are "using" the Navigator Metadata Server database in your database server client.

 

With that information, we can see if they may be out of sync and we can decide if we need to adjust the values.

Re: Navigator metadata server keep on crashing after changing to Mysql DB

New Contributor

Hi @bgooley ,

 

(1)rpm -qa | grep cloudera gives the following output:


cloudera-manager-agent-5.13.1-1.cm5131.p0.2.el7.x86_64
cloudera-manager-daemons-5.13.1-1.cm5131.p0.2.el7.x86_64
cloudera-manager-server-5.13.1-1.cm5131.p0.2.el7.x86_64

 

(2)And the query result is:


ordinal | old_ordinal
---------+-------------
       26 |             26

 

Thanks

Re: Navigator metadata server keep on crashing after changing to Mysql DB

New Contributor
So... what happened next? I also have this issue, and you leave me hanging!

Re: Navigator metadata server keep on crashing after changing to Mysql DB

New Contributor

I had to reset the Navigator metadata server and followed the steps below:

1) Stop Navigator Metadata Server from CM

2) Take a back up of Navigator Metadata Server Storage Dir (if you want to archive the old data)

3) mv <storage_dir> < new_location>

4) Back up your navms database

5) In NMS database run:

"DELETE FROM NAV_UPGRADE_ORDINAL;" "INSERT INTO NAV_UPGRADE_ORDINAL VALUES (-1, -1);"

6) Start Navigator Metadata Server

 

this will delete all metadata, custom data added in the Navigator UI but does not affect the Audit logs visible under Navigator UI -> Audits.

Re: Navigator metadata server keep on crashing after changing to Mysql DB

New Contributor

Sadly this doesn't work for me- my Navigator DB doesn't have this table.

But I'm not upgrading - I'm using oracle in the first place.

Maybe it's another kind of installation error?

Or do I need a software upgrade?

 

$ rpm -qa | grep cloudera
cloudera-manager-agent-5.12.1-1.cm5121.p0.6.el6.x86_64
cloudera-manager-daemons-5.12.1-1.cm5121.p0.6.el6.x86_64
cloudera-manager-server-5.12.1-1.cm5121.p0.6.el6.x86_64

 

3:33:19.872 PM ERROR SolrCore
[qtp1384454980-63]: org.apache.solr.common.SolrException: Cursor functionality requires a sort containing a uniqueKey field tie breaker
at org.apache.solr.search.CursorMark.<init>(CursorMark.java:104)