question Re: Navigator metadata server keep on crashing after changing to Mysql DB in Archives of Support Questions (Read Only)

Navigator metadata server keep on crashing after changing to Mysql DB

ronnie10 — Wed, 20 Sep 2017 06:59:17 GMT

Hi there,

I have enabled Navigator metadata server to use mysql db, but it keep on crashing with solr error, hope someone can help on this.

Thanks

SolrCore

[qtp2123908275-75]: org.apache.solr.common.SolrException: Cursor functionality requires a sort containing a uniqueKey field tie breaker
	at org.apache.solr.search.CursorMark.<init>(CursorMark.java:104)
	at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:179)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1476)
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:256)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
	at com.cloudera.nav.auth.SharedSecretRequestFilter.doFilter(SharedSecretRequestFilter.java:55)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:370)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:745)

Re: Navigator metadata server keep on crashing after changing to Mysql DB

elifsurmeli — Tue, 30 Jan 2018 12:25:25 GMT

Hi, can you find a solution for this issue?

Thanks

Re: Navigator metadata server keep on crashing after changing to Mysql DB

bgooley — Tue, 30 Jan 2018 19:41:05 GMT

@elifsurmeli,

This usually indicates that the Solr data is out of sync (version-wise) with what nav_upgrade_ordinal table in your Navigator Metadata Server database.

Please do the following to see if that is the cause:

(1) on the Navigator Metadata host, run rpm -qa |grep cloudera and post the results (so we can confirm the version of Navigator

(2) Run the following query in your Navigator Metadata Server database (via your favorite db tool):

select * from NAV_UPGRADE_ORDINAL;

NOTE: The above assumes that you are "using" the Navigator Metadata Server database in your database server client.

With that information, we can see if they may be out of sync and we can decide if we need to adjust the values.

Re: Navigator metadata server keep on crashing after changing to Mysql DB

elifsurmeli — Fri, 09 Feb 2018 12:41:46 GMT

Hi @bgooley ,

(1)rpm -qa | grep cloudera gives the following output:

cloudera-manager-agent-5.13.1-1.cm5131.p0.2.el7.x86_64
cloudera-manager-daemons-5.13.1-1.cm5131.p0.2.el7.x86_64
cloudera-manager-server-5.13.1-1.cm5131.p0.2.el7.x86_64

(2)And the query result is:

ordinal | old_ordinal
---------+-------------
26 | 26

Thanks

Re: Navigator metadata server keep on crashing after changing to Mysql DB

JohnDB — Thu, 19 Apr 2018 10:55:21 GMT

So... what happened next? I also have this issue, and you leave me hanging!

Re: Navigator metadata server keep on crashing after changing to Mysql DB

elifsurmeli — Thu, 19 Apr 2018 15:55:44 GMT

I had to reset the Navigator metadata server and followed the steps below:

1) Stop Navigator Metadata Server from CM

2) Take a back up of Navigator Metadata Server Storage Dir (if you want to archive the old data)

3) mv <storage_dir> < new_location>

4) Back up your navms database

5) In NMS database run:

"DELETE FROM NAV_UPGRADE_ORDINAL;" "INSERT INTO NAV_UPGRADE_ORDINAL VALUES (-1, -1);"

6) Start Navigator Metadata Server

this will delete all metadata, custom data added in the Navigator UI but does not affect the Audit logs visible under Navigator UI -> Audits.

Re: Navigator metadata server keep on crashing after changing to Mysql DB

JohnDB — Mon, 23 Apr 2018 16:23:04 GMT

Sadly this doesn't work for me- my Navigator DB doesn't have this table.

But I'm not upgrading - I'm using oracle in the first place.

Maybe it's another kind of installation error?

Or do I need a software upgrade?

$ rpm -qa | grep cloudera
cloudera-manager-agent-5.12.1-1.cm5121.p0.6.el6.x86_64
cloudera-manager-daemons-5.12.1-1.cm5121.p0.6.el6.x86_64
cloudera-manager-server-5.12.1-1.cm5121.p0.6.el6.x86_64

3:33:19.872 PM ERROR SolrCore
[qtp1384454980-63]: org.apache.solr.common.SolrException: Cursor functionality requires a sort containing a uniqueKey field tie breaker
at org.apache.solr.search.CursorMark.<init>(CursorMark.java:104)