Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Specified config does not exist in ZooKeeper (Log Search is broken)

avatar
Expert Contributor

Fresh cluster (2.6.4) was working, restarted zookeeper and now seeing multiple errors. How can I fix this and why doesn't Solr just send new data to zookeeper?

solr-problem.png

One Log Example:

null:org.apache.solr.common.SolrException: SolrCore 'hadoop_logs_shard0_replica1' is not available due to init failure: Specified config does not exist in ZooKeeper: hadoop_logs
	at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1071)
	at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:252)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:414)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:499)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.cloud.ZooKeeperException: Specified config does not exist in ZooKeeper: hadoop_logs
	at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:161)
	at org.apache.solr.cloud.CloudConfigSetService.createCoreResourceLoader(CloudConfigSetService.java:36)
	at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:75)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:818)
	at org.apache.solr.core.CoreContainer.access$000(CoreContainer.java:90)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:473)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:464)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more
1 ACCEPTED SOLUTION

avatar
Expert Contributor

For those facing this issue with hadoop_logs (Log Search)

  • Delete the Log Search service from Ambari UI
  • Go into the zookeeper shell /usr/hdp/current/zookeeper-client/bin/zkCli.sh
  • rmr /infra-solr/configs/hadoop_logs
  • Open up solr and delete the shards/collection for hadoop_logs
  • SSH into your SOLR servers as root/sudo
  • Delete /opt/ambari_infra_solr/data/hadoop_logs*
  • Make sure you have at least 2 instances of SOLR installed
  • Install Log Search on a new node (a SOLR instance should be installed on this node as well)
  • Log search should be running now

View solution in original post

1 REPLY 1

avatar
Expert Contributor

For those facing this issue with hadoop_logs (Log Search)

  • Delete the Log Search service from Ambari UI
  • Go into the zookeeper shell /usr/hdp/current/zookeeper-client/bin/zkCli.sh
  • rmr /infra-solr/configs/hadoop_logs
  • Open up solr and delete the shards/collection for hadoop_logs
  • SSH into your SOLR servers as root/sudo
  • Delete /opt/ambari_infra_solr/data/hadoop_logs*
  • Make sure you have at least 2 instances of SOLR installed
  • Install Log Search on a new node (a SOLR instance should be installed on this node as well)
  • Log search should be running now