Created on 03-23-2017 12:41 AM - edited 09-16-2022 04:19 AM
Hello
We use solr: 4.10.3-cdh5.4.9
And periodically 'cretecollection' command timeouts with such stacktrase:
14:35:58 Error: A call to SolrCloud WEB APIs failed: HTTP/1.1 500 Internal Server Error 14:35:58 Server: nginx 14:35:58 Date: Tue, 21 Mar 2017 11:20:25 GMT 14:35:58 Content-Type: application/xml;charset=UTF-8 14:35:58 Transfer-Encoding: chunked 14:35:58 Connection: keep-alive 14:35:58 X-Market-Req-ID: 1490095045005/30f3544fc6f139218869df711c4e19f0 14:35:58 14:35:58 <?xml version="1.0" encoding="UTF-8"?> 14:35:58 14:35:58 <response> 14:35:58 14:35:58 <lst name="responseHeader"> 14:35:58 <int name="status"> 14:35:58 500</int> 14:35:58 <int name="QTime"> 14:35:58 180008</int> 14:35:58 </lst> 14:35:58 <lst name="error"> 14:35:58 <str name="msg"> 14:35:58 createcollection the collection time out:180s</str> 14:35:58 <str name="trace"> 14:35:58 org.apache.solr.common.SolrException: createcollection the collection time out:180s 14:35:58 at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:364) 14:35:58 at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:316) 14:35:58 at org.apache.solr.handler.admin.CollectionsHandler.handleCreateAction(CollectionsHandler.java:482) 14:35:58 at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:153) 14:35:58 at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) 14:35:58 at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:770) 14:35:58 at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:271) 14:35:58 at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:211) 14:35:58 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) 14:35:58 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) 14:35:58 at org.apache.solr.servlet.SolrHadoopAuthenticationFilter$2.doFilter(SolrHadoopAuthenticationFilter.java:353) 14:35:58 at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592) 14:35:58 at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291) 14:35:58 at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:555) 14:35:58 at org.apache.solr.servlet.SolrHadoopAuthenticationFilter.doFilter(SolrHadoopAuthenticationFilter.java:358) 14:35:58 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) 14:35:58 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) 14:35:58 at org.apache.solr.servlet.HostnameFilter.doFilter(HostnameFilter.java:86) 14:35:58 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) 14:35:58 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) 14:35:58 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) 14:35:58 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) 14:35:58 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) 14:35:58 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) 14:35:58 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) 14:35:58 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) 14:35:58 at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861) 14:35:58 at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:620) 14:35:58 at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) 14:35:58 at java.lang.Thread.run(Thread.java:745) 14:35:58 </str> 14:35:58 <int name="code"> 14:35:58 500</int> 14:35:58 </lst> 14:35:58 14:35:58 </response>
Also, I have created bug in apache: https://issues.apache.org/jira/browse/SOLR-10334
Created 03-23-2017 02:46 AM
I did encounter such a situation from time to time in the past (not recently).
Each time, it was because at least one of the Solr server was busy doing something else.
Check that each Solr server has good health and that there is no defect on the health of the other collections.