I am trying to re-create a ranger-audits collection for Infra-Solr with numShards=2 instead of 1.     # curl -k --negotiate -u : "https://fqdn.example.net:8995/solr/admin/collections?action=CREATE&name=ranger_audits&collection.configName=ranger_audits&numShards=2" { "responseHeader":{ "status":400, "QTime":1730}, "failure":{ "fqdn.example.net:8995_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at https://ufqdn.example.net:8995/solr: Error CREATEing SolrCore 'ranger_audits_shard2_replica_n2': Unable to create core [ranger_audits_shard2_replica_n2] Caused by: Could not initialize class java.time.zone.ZoneRulesProvider"}, "success":{ "fqdn.example.net:8995_solr":{ "responseHeader":{ "status":0, "QTime":1278}, "core":"ranger_audits_shard1_replica_n1"}}, "Operation create caused exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Underlying core creation failed while creating collection: ranger_audits", "exception":{ "msg":"Underlying core creation failed while creating collection: ranger_audits", "rspCode":400}, "error":{ "metadata":[ "error-class","org.apache.solr.common.SolrException", "root-error-class","org.apache.solr.common.SolrException"], "msg":"Underlying core creation failed while creating collection: ranger_audits", "code":400}}     Re-creating the collection with numShards=1 does not create any issues and it is executed correctly.   curl -k --negotiate -u : "https://fqdn.example.com:8995/solr/admin/collections?action=CREATE&name=ranger_audits&collection.configName=ranger_audits&numShards=1"   At the same time, in Solr logs I receive this error:   [c:ranger_audits s:shard2 r:core_node4 x:ranger_audits_shard2_replica_n2] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error CREATEing SolrCore 'ranger_audits_shard2_replica_n2': Unable to create core [ranger_audits_shard2_replica_n2] Caused by: Could not initialize class java.time.zone.ZoneRulesProvider at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1260) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:821) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:787) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:538) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:414) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:346) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.solr.common.SolrException: Unable to create core [ranger_audits_shard2_replica_n2] at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1354) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1221) ... 50 more Caused by: org.apache.solr.common.SolrException: Could not initialize class java.time.zone.ZoneRulesProvider at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1084) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:905) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1333) ... 51 more Caused by: java.lang.NoClassDefFoundError: Could not initialize class java.time.zone.ZoneRulesProvider at java.time.format.DateTimeFormatterBuilder$ZoneTextPrinterParser.getTree(DateTimeFormatterBuilder.java:3726) at java.time.format.DateTimeFormatterBuilder$ZoneIdPrinterParser.parse(DateTimeFormatterBuilder.java:3856) at java.time.format.DateTimeFormatterBuilder$CompositePrinterParser.parse(DateTimeFormatterBuilder.java:2209) at java.time.format.DateTimeFormatterBuilder$CompositePrinterParser.parse(DateTimeFormatterBuilder.java:2209) at java.time.format.DateTimeFormatterBuilder$CompositePrinterParser.parse(DateTimeFormatterBuilder.java:2209) at java.time.format.DateTimeFormatterBuilder$CompositePrinterParser.parse(DateTimeFormatterBuilder.java:2219) at java.time.format.DateTimeFormatter.parseUnresolved0(DateTimeFormatter.java:2010) at java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:1939) at java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1819) at org.apache.solr.update.processor.ParseDateFieldUpdateProcessorFactory.parseInstant(ParseDateFieldUpdateProcessorFactory.java:230) at org.apache.solr.update.processor.ParseDateFieldUpdateProcessorFactory.validateFormatter(ParseDateFieldUpdateProcessorFactory.java:214) at org.apache.solr.update.processor.ParseDateFieldUpdateProcessorFactory.init(ParseDateFieldUpdateProcessorFactory.java:189) at org.apache.solr.core.PluginBag.initInstance(PluginBag.java:106) at org.apache.solr.core.PluginBag.createPlugin(PluginBag.java:148) at org.apache.solr.core.PluginBag.init(PluginBag.java:290) at org.apache.solr.core.PluginBag.init(PluginBag.java:279) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1008) ... 53 more   Cluster is kerberized and cloudera runtime 7.1.6-1.   Can anyone can give me a hint on how to fix this issue?   ... View more

After upgrading from Ambari 2.5.0.3 to 2.6.2.2, ambari-server.log file suddenly start recording following entries:   28 Nov 2020 16:48:52,372 WARN [ambari-action-scheduler] ActionScheduler:316 - Exception received java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat7 could not be found at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253) at org.apache.ambari.server.actionmanager.ActionScheduler.processInProgressStage(ActionScheduler.java:704) at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:417) at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat7 could not be found at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422) at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247) ... 4 more     Server's OS has always been RedHat 7.x and prior to the upgrade there were no issues with the OS version:     $ cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.5 (Maipo)     To make this more interesting, Ambari WebUI in the hosts details correctly identifies the OS version     OS: redhat7 (x86_64)     Ambari WebUI and `ambari-server --version` already show upgraded version `2.6.2.2`.   Please help solve this issue. ... View more