Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CM / CDH 6.2.X w/ PSQL:Cursor functionality requires a sort containing a uniqueKey field tie breaker

CM / CDH 6.2.X w/ PSQL:Cursor functionality requires a sort containing a uniqueKey field tie breaker

Explorer

Hey All,

 

I'm installing a vanilla CM / CDH 6.2.X and running into the issue below from the Navigator Metadata Server:

 

10:45:54.757 PM	INFO	JsonModelLoader	
[main]: Added audit event model for SENTRY
10:45:54.761 PM	INFO	JsonModelLoader	
[main]: Added audit event model for SOLR
10:46:10.631 PM	WARN	SecurityHandler	
[main]: ServletContext@o.e.j.w.WebAppContext@241e8ea6{/solr,[file:///var/lib/cloudera-scm-navigator/temp/jetty-0.0.0.0-7187-solr-4.10.3.war-_solr-any-4050545960767447337.dir/webinf/, file:///var/lib/cloudera-scm-navigator/temp/jetty-0.0.0.0-7187-solr-4.10.3.war-_solr-any-4050545960767447337.dir/webapp/],STARTING}{/opt/cloudera/cm/cloudera-navigator-server/wars/solr-4.10.3.war} has uncovered http methods for path: /*
10:46:12.171 PM	WARN	IndexSchema	
[coreLoadExecutor-5-thread-1]: Field user is not multivalued and destination for multiple copyFields (2)
10:46:14.883 PM	INFO	NavServerUtil	
[main]: Found 4 documents in solr core nav_elements
10:46:14.895 PM	INFO	NavServerUtil	
[main]: Found 1 documents in solr core nav_relations
10:46:14.896 PM	INFO	SolrSchemaUpgrade	
[main]: Checking if solr schema upgrade is needed.
10:46:14.928 PM	INFO	SolrSchemaUpgrade	
[main]: Schema version 2900 already up to date with latest schema 2900.
10:46:14.956 PM	INFO	SolrSchemaUpgrade	
[main]: Current database upgrade ordinal is -1, latest upgrade step has ordinal 28
10:46:15.024 PM	INFO	AbstractSolrUpgrade	
[main]: Starting upgrade step: com.cloudera.nav.server.upgrade.DeleteEntities : Delete temporary entities.
10:46:15.031 PM	INFO	NavServerUtil	
[main]: Found 4 documents in solr core nav_elements
10:46:15.039 PM	INFO	NavServerUtil	
[main]: Found 1 documents in solr core nav_relations
10:46:15.042 PM	INFO	DeleteEntities	
[main]: Delete temporary relations query: {!join from=identity to=endpoint1Ids fromIndex=nav_elements}fileSystemPath:(*\/_SUCCESS OR *\/.trash OR \/user\/history OR *\/.staging OR \/tmp\/logs) AND sourceType:HDFS
10:46:15.206 PM	ERROR	SolrCore	
[qtp1100399456-26]: org.apache.solr.common.SolrException: Cursor functionality requires a sort containing a uniqueKey field tie breaker
	at org.apache.solr.search.CursorMark.<init>(CursorMark.java:104)
	at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:179)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
	at com.cloudera.nav.auth.SharedSecretRequestFilter.doFilter(SharedSecretRequestFilter.java:55)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:502)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
	at java.lang.Thread.run(Thread.java:748)

10:46:15.221 PM	ERROR	NavServer	
[main]: Error upgrading solr configuration or data.
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Cursor functionality requires a sort containing a uniqueKey field tie breaker
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:310)
	at com.cloudera.nav.utils.solr.SolrResultSetIterator.getNextBatch(SolrResultSetIterator.java:96)
	at com.cloudera.nav.utils.solr.SolrResultSetIterator.hasNext(SolrResultSetIterator.java:76)
	at com.cloudera.nav.server.upgrade.DeleteEntities.deleteForSide(DeleteEntities.java:248)
	at com.cloudera.nav.server.upgrade.DeleteEntities.blindDelete(DeleteEntities.java:223)
	at com.cloudera.nav.server.upgrade.DeleteEntities.upgrade(DeleteEntities.java:76)
	at com.cloudera.nav.server.upgrade.AbstractSolrUpgrade.upgrade(AbstractSolrUpgrade.java:156)
	at com.cloudera.nav.server.SolrSchemaUpgrade.performJavaUpgrade(SolrSchemaUpgrade.java:400)
	at com.cloudera.nav.server.SolrSchemaUpgrade.upgradeSolr(SolrSchemaUpgrade.java:292)
	at com.cloudera.nav.server.NavServer.run(NavServer.java:434)
	at com.cloudera.nav.server.NavServer.main(NavServer.java:123)
10:46:15.329 PM	INFO	NavigatorInfrastructureEventDispatcher	
[pool-12-thread-1]: Waiting for Solr to Start.
10:46:15.331 PM	INFO	NavigatorInfrastructureEventDispatcher	
[pool-12-thread-1]: Succesfully started Solr.
10:46:15.332 PM	INFO	NavigatorInfrastructureEventDispatcher	
[pool-12-thread-1]: Waiting for Solr upgrade to complete.
10:46:15.489 PM	ERROR	NavServerUtil	
[pool-12-thread-1]: Error encountered in fetching the udate progress information.Please enable TRACE level logging for SQL exception details.
10:46:15.496 PM	INFO	NavServerUtil	
[pool-12-thread-1]: Exception occurred when checking for upgrade status.
java.lang.RuntimeException: Error encountered in fetching the udate progress information.
	at com.cloudera.nav.persist.PersistUtils.handleDataAccessException(PersistUtils.java:88)
	at com.cloudera.nav.server.NavServerUtil.getUpgradeProgressInfo(NavServerUtil.java:281)
	at com.cloudera.nav.server.NavServerUtil.isUpgradeDone(NavServerUtil.java:231)
	at com.cloudera.nav.utils.UpgradeCompletionChecker.isUpgradeDone(UpgradeCompletionChecker.java:13)
	at com.cloudera.nav.utils.NavigatorInfrastructureEventDispatcher$SolrUpgradePoller.isUpgradeDone(NavigatorInfrastructureEventDispatcher.java:158)
	at com.cloudera.nav.server.SolrMaintenanceAware.waitForUpgrade(SolrMaintenanceAware.java:23)
	at com.cloudera.nav.utils.NavigatorInfrastructureEventDispatcher$SolrUpgradePoller.run(NavigatorInfrastructureEventDispatcher.java:153)
	at com.cloudera.nav.utils.NavigatorInfrastructureEventDispatcher.dispatchSolrUpgradedEvent(NavigatorInfrastructureEventDispatcher.java:107)
	at com.cloudera.nav.utils.NavigatorInfrastructureEventDispatcher.access$300(NavigatorInfrastructureEventDispatcher.java:35)
	at com.cloudera.nav.utils.NavigatorInfrastructureEventDispatcher$3.run(NavigatorInfrastructureEventDispatcher.java:93)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
10:46:15.506 PM	INFO	SolrMaintenanceAware	
[pool-12-thread-1]: Checking for upgrade done status to be set.
10:46:15.514 PM	ERROR	NavServerUtil	
[pool-12-thread-1]: Error encountered in fetching the udate progress information.Please enable TRACE level logging for SQL exception details.
10:46:15.514 PM	INFO	NavServerUtil	
[pool-12-thread-1]: Exception occurred when checking for upgrade status.
java.lang.RuntimeException: Error encountered in fetching the udate progress information.
	at com.cloudera.nav.persist.PersistUtils.handleDataAccessException(PersistUtils.java:88)
	at com.cloudera.nav.server.NavServerUtil.getUpgradeProgressInfo(NavServerUtil.java:281)
	at com.cloudera.nav.server.NavServerUtil.isUpgradeDone(NavServerUtil.java:231)
	at com.cloudera.nav.utils.UpgradeCompletionChecker.isUpgradeDone(UpgradeCompletionChecker.java:13)
	at com.cloudera.nav.utils.NavigatorInfrastructureEventDispatcher$SolrUpgradePoller.isUpgradeDone(NavigatorInfrastructureEventDispatcher.java:158)
	at com.cloudera.nav.server.SolrMaintenanceAware.pollForUpgradeDone(SolrMaintenanceAware.java:34)
	at com.cloudera.nav.server.SolrMaintenanceAware.waitForUpgrade(SolrMaintenanceAware.java:24)
	at com.cloudera.nav.utils.NavigatorInfrastructureEventDispatcher$SolrUpgradePoller.run(NavigatorInfrastructureEventDispatcher.java:153)
	at com.cloudera.nav.utils.NavigatorInfrastructureEventDispatcher.dispatchSolrUpgradedEvent(NavigatorInfrastructureEventDispatcher.java:107)
	at com.cloudera.nav.utils.NavigatorInfrastructureEventDispatcher.access$300(NavigatorInfrastructureEventDispatcher.java:35)
	at com.cloudera.nav.utils.NavigatorInfrastructureEventDispatcher$3.run(NavigatorInfrastructureEventDispatcher.java:93)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
10:46:15.516 PM	INFO	SolrMaintenanceAware	
[pool-12-thread-1]: Upgrade status not set or available, continue checking for upgrade status to be set.

My backend is a PostgreSQL database.  Navigator Metastore Server keeps exiting and restarting as a result:

 

Unexpected Exits                                     Suppress...
This role encountered 4 unexpected exit(s) in the previous 5 minute(s). Critical threshold: any.
 
How could I solve this?  The other posts and solutions I could find were specific to MySQL only.  
 
Also only the servers are reporting and the Cloudera Management Service is installed.  Nothing else is installed on this Cluster yet.  

 

Thx,
TK

2 REPLIES 2

Re: CM / CDH 6.2.X w/ PSQL:Cursor functionality requires a sort containing a uniqueKey field tie bre

Champion

Hi 

Take a peek in to the below , please try in your lower enviroment and validate before moving to production box. 

 

https://community.cloudera.com/t5/Data-Discovery-Optimization/Navigator-metadata-server-keep-on-cras...

Highlighted

Re: CM / CDH 6.2.X w/ PSQL:Cursor functionality requires a sort containing a uniqueKey field tie bre

Explorer

Thank you but I saw that already before posting and tried the solutions from it.  However, I don't have these tables in PostgreSQL on CDH / CM 6.2.X:

 

"DELETE FROM NAV_UPGRADE_ORDINAL;" "INSERT INTO NAV_UPGRADE_ORDINAL VALUES (-1, -1);"

 

select * from NAV_UPGRADE_ORDINAL;

 

So I posted since on first impressions, this appears a unique case to 6.2.X and PostgreSQL.

 

postgres=# \c nav_mws01
Password for user postgres:
You are now connected to database "nav_mws01" as user "postgres".
nav_mws01=# \dt
List of relations
Schema | Name | Type | Owner
--------+--------------------------------+-------+-----------
public | activity_audit | table | nav_mws01
public | activity_last_poll_time | table | nav_mws01
public | audit_schema_version | table | nav_mws01
public | hbase_audit_events | table | nav_mws01
public | hbase_audit_events_2019_04_14 | table | nav_mws01
public | hbase_audit_events_2019_04_15 | table | nav_mws01
public | hbase_audit_events_2019_04_26 | table | nav_mws01
public | hbase_audit_events_2019_04_27 | table | nav_mws01
public | hbase_audit_events_2019_04_28 | table | nav_mws01
public | hdfs_audit_events | table | nav_mws01
public | hdfs_audit_events_2019_04_14 | table | nav_mws01
public | hdfs_audit_events_2019_04_15 | table | nav_mws01
public | hdfs_audit_events_2019_04_26 | table | nav_mws01
public | hdfs_audit_events_2019_04_27 | table | nav_mws01
public | hdfs_audit_events_2019_04_28 | table | nav_mws01
public | hive_audit_events | table | nav_mws01
public | hive_audit_events_2019_04_14 | table | nav_mws01
public | hive_audit_events_2019_04_15 | table | nav_mws01
public | hive_audit_events_2019_04_26 | table | nav_mws01
public | hive_audit_events_2019_04_27 | table | nav_mws01
public | hive_audit_events_2019_04_28 | table | nav_mws01
public | hue_audit_events | table | nav_mws01
public | hue_audit_events_2019_04_14 | table | nav_mws01
public | hue_audit_events_2019_04_15 | table | nav_mws01
public | hue_audit_events_2019_04_26 | table | nav_mws01
public | hue_audit_events_2019_04_27 | table | nav_mws01
public | hue_audit_events_2019_04_28 | table | nav_mws01
public | impala_audit_events | table | nav_mws01
public | impala_audit_events_2019_04_14 | table | nav_mws01
public | impala_audit_events_2019_04_15 | table | nav_mws01
public | impala_audit_events_2019_04_26 | table | nav_mws01
public | impala_audit_events_2019_04_27 | table | nav_mws01
public | impala_audit_events_2019_04_28 | table | nav_mws01
public | nav_count_staging_raw | table | nav_mws01
public | nav_file_info | table | nav_mws01
public | nav_file_info_staging | table | nav_mws01
public | nav_file_reads | table | nav_mws01
public | nav_file_reads_staging | table | nav_mws01
public | nav_file_reads_summary | table | nav_mws01
public | nav_parent_info | table | nav_mws01
public | nav_parent_info_staging | table | nav_mws01
public | nav_service_info | table | nav_mws01
public | nav_service_info_staging | table | nav_mws01
public | navms_audit_events | table | nav_mws01
public | navms_audit_events_2019_04_14 | table | nav_mws01
public | navms_audit_events_2019_04_15 | table | nav_mws01
public | navms_audit_events_2019_04_26 | table | nav_mws01
public | navms_audit_events_2019_04_27 | table | nav_mws01
public | navms_audit_events_2019_04_28 | table | nav_mws01
public | partition_info | table | nav_mws01
public | sentry_audit_events | table | nav_mws01
public | sentry_audit_events_2019_04_14 | table | nav_mws01
public | sentry_audit_events_2019_04_15 | table | nav_mws01
public | sentry_audit_events_2019_04_26 | table | nav_mws01
public | sentry_audit_events_2019_04_27 | table | nav_mws01
public | sentry_audit_events_2019_04_28 | table | nav_mws01
public | solr_audit_events | table | nav_mws01
public | solr_audit_events_2019_04_14 | table | nav_mws01
public | solr_audit_events_2019_04_15 | table | nav_mws01
public | solr_audit_events_2019_04_26 | table | nav_mws01
public | solr_audit_events_2019_04_27 | table | nav_mws01
public | solr_audit_events_2019_04_28 | table | nav_mws01
(62 rows)

nav_mws01=# select * from NAV_UPGRADE_ORDINAL;
ERROR: relation "nav_upgrade_ordinal" does not exist
LINE 1: select * from NAV_UPGRADE_ORDINAL;
^
nav_mws01=#

 

Thx,
TK