Why my Solr service is down, i got it restart and after 20s it'll be down without any log error

[#1]

I'm just get started in setting up a simple cluster with 1 master and 4 node, i'm choose the "All service" package and then using the default setting when install. Then got all of my service is in good health except bad health for the Solr service, i try checking the log find but don't see any warning.

Have annyone meet the same situation, thanks for the help

[#2]

Hi All

I have the same problem. I've seen the log and I don't see something.

CDH 5.3.1

2014-12-02 12:03:04,837 INFO org.apache.solr.servlet.ProxyUserFilter: Loading proxyuser settings [solr.security.proxyuser.hue.hosts]=[*]
2014-12-02 12:03:04,837 INFO org.apache.solr.servlet.ProxyUserFilter: Loading proxyuser settings [solr.security.proxyuser.hue.groups]=[*]
2014-12-02 12:03:04,953 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2014-12-02 12:03:04,981 WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter: 'signature.secret' configuration not set, using a random value as secret
2014-12-02 12:03:05,028 INFO org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:
2014-12-02 12:03:05,041 INFO org.apache.solr.servlet.SolrDispatchFilter: SolrDispatchFilter.init()
2014-12-02 12:03:05,084 INFO org.apache.solr.core.SolrResourceLoader: No /solr/home in JNDI
2014-12-02 12:03:05,084 INFO org.apache.solr.core.SolrResourceLoader: using system property solr.solr.home: /var/lib/solr
2014-12-02 12:03:05,084 INFO org.apache.solr.core.SolrResourceLoader: new SolrResourceLoader for directory: '/var/lib/solr/'
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-cdh5.1.3--1, built on 09/17/2014 02:09 GMT
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=hd02
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.7.0_55
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-7-oracle-cloudera/jre
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/opt/cloudera/parcels/CDH-5.1.3-1.cdh5.1.3.p0.12/lib/bigtop-tomcat/bin/bootstrap.jar
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/var/lib/solr/
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=3.2.0-4-amd64
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=solr
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/var/run/solr
2014-12-02 12:03:05,308 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/run/cloudera-scm-agent/process/984-solr-SOLR_SERVER
2014-12-02 12:03:05,309 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=hd01:2181/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6c30aec7
2014-12-02 12:03:05,326 INFO org.apache.solr.common.cloud.ConnectionManager: Waiting for client to connect to ZooKeeper
2014-12-02 12:03:05,329 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server hd01/192.168.20.1:2181. Will not attempt to authenticate using SASL (unknown error)
2014-12-02 12:03:05,334 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to hd01/192.168.20.1:2181, initiating session
2014-12-02 12:03:05,345 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server hd01/192.168.20.1:2181, sessionid = 0x14a0a9b2d300066, negotiated timeout = 30000
2014-12-02 12:03:05,348 INFO org.apache.solr.common.cloud.ConnectionManager: Watcher org.apache.solr.common.cloud.ConnectionManager@6c30aec7 name:ZooKeeperConnection Watcher:hd01:2181/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
2014-12-02 12:03:05,348 INFO org.apache.solr.common.cloud.ConnectionManager: Client is connected to ZooKeeper
2014-12-02 12:03:05,353 INFO org.apache.solr.core.ConfigSolr: Loading solr.xml from ZooKeeper
2014-12-02 12:03:05,459 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr
2014-12-02 12:03:05,459 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment
2014-12-02 12:03:05,459 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps
2014-12-02 12:03:05,460 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr
2014-12-02 12:03:05,460 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/js
2014-12-02 12:03:05,460 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/js/scripts
2014-12-02 12:03:05,461 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/js/lib
2014-12-02 12:03:05,461 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/WEB-INF
2014-12-02 12:03:05,461 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/WEB-INF/lib
2014-12-02 12:03:05,464 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/css
2014-12-02 12:03:05,464 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/css/styles
2014-12-02 12:03:05,465 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/img
2014-12-02 12:03:05,465 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/img/ico
2014-12-02 12:03:05,468 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/img/filetypes
2014-12-02 12:03:05,471 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/META-INF
2014-12-02 12:03:05,471 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/solr/tpl
2014-12-02 12:03:05,472 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/webapps/ROOT
2014-12-02 12:03:05,472 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/work
2014-12-02 12:03:05,472 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/work/Catalina
2014-12-02 12:03:05,472 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/work/Catalina/localhost
2014-12-02 12:03:05,472 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/work/Catalina/localhost/solr
2014-12-02 12:03:05,472 INFO org.apache.solr.core.SolrCoreDiscoverer: Looking for cores in /var/lib/solr/tomcat-deployment/conf
2014-12-02 12:03:05,480 INFO org.apache.zookeeper.ZooKeeper: Session: 0x14a0a9b2d300066 closed
2014-12-02 12:03:05,480 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2014-12-02 12:03:05,487 INFO org.apache.solr.core.CoreContainer: New CoreContainer 1045865347
2014-12-02 12:03:05,487 INFO org.apache.solr.core.CoreContainer: Loading cores into CoreContainer [instanceDir=/var/lib/solr/]
2014-12-02 12:03:05,508 INFO org.apache.solr.handler.component.HttpShardHandlerFactory: Setting socketTimeout to: 0
2014-12-02 12:03:05,508 INFO org.apache.solr.handler.component.HttpShardHandlerFactory: Setting urlScheme to: http://
2014-12-02 12:03:05,508 INFO org.apache.solr.handler.component.HttpShardHandlerFactory: Setting connTimeout to: 0
2014-12-02 12:03:05,508 INFO org.apache.solr.handler.component.HttpShardHandlerFactory: Setting maxConnectionsPerHost to: 20
2014-12-02 12:03:05,508 INFO org.apache.solr.handler.component.HttpShardHandlerFactory: Setting corePoolSize to: 0
2014-12-02 12:03:05,508 INFO org.apache.solr.handler.component.HttpShardHandlerFactory: Setting maximumPoolSize to: 2147483647
2014-12-02 12:03:05,508 INFO org.apache.solr.handler.component.HttpShardHandlerFactory: Setting maxThreadIdleTime to: 5
2014-12-02 12:03:05,508 INFO org.apache.solr.handler.component.HttpShardHandlerFactory: Setting sizeOfQueue to: -1
2014-12-02 12:03:05,508 INFO org.apache.solr.handler.component.HttpShardHandlerFactory: Setting fairnessPolicy to: false
2014-12-02 12:03:05,509 INFO org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
2014-12-02 12:03:05,654 INFO org.apache.solr.logging.LogWatcher: Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
2014-12-02 12:03:05,655 INFO org.apache.solr.core.ZkContainer: Zookeeper client=hd01:2181/solr
2014-12-02 12:03:05,664 INFO org.apache.solr.cloud.ZkController: zkHost includes chroot
2014-12-02 12:03:05,664 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=hd01:2181 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1b878da9
2014-12-02 12:03:05,665 INFO org.apache.solr.common.cloud.ConnectionManager: Waiting for client to connect to ZooKeeper
2014-12-02 12:03:05,666 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server hd01/192.168.20.1:2181. Will not attempt to authenticate using SASL (unknown error)
2014-12-02 12:03:05,666 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to hd01/192.168.20.1:2181, initiating session
2014-12-02 12:03:05,677 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server hd01/192.168.20.1:2181, sessionid = 0x14a0a9b2d30006d, negotiated timeout = 60000
2014-12-02 12:03:05,678 INFO org.apache.solr.common.cloud.ConnectionManager: Watcher org.apache.solr.common.cloud.ConnectionManager@1b878da9 name:ZooKeeperConnection Watcher:hd01:2181 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
2014-12-02 12:03:05,678 INFO org.apache.solr.common.cloud.ConnectionManager: Client is connected to ZooKeeper
2014-12-02 12:03:05,689 INFO org.apache.zookeeper.ZooKeeper: Session: 0x14a0a9b2d30006d closed
2014-12-02 12:03:05,689 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2014-12-02 12:03:05,690 INFO org.apache.solr.client.solrj.impl.HttpClientUtil: Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=0&connTimeout=0
2014-12-02 12:03:05,691 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=hd01:2181/solr sessionTimeout=15000 watcher=org.apache.solr.common.cloud.ConnectionManager@2c1a4849
2014-12-02 12:03:05,692 INFO org.apache.solr.common.cloud.ConnectionManager: Waiting for client to connect to ZooKeeper
2014-12-02 12:03:05,692 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server hd01/192.168.20.1:2181. Will not attempt to authenticate using SASL (unknown error)
2014-12-02 12:03:05,692 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to hd01/192.168.20.1:2181, initiating session
2014-12-02 12:03:05,700 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server hd01/192.168.20.1:2181, sessionid = 0x14a0a9b2d30006f, negotiated timeout = 15000
2014-12-02 12:03:05,700 INFO org.apache.solr.common.cloud.ConnectionManager: Watcher org.apache.solr.common.cloud.ConnectionManager@2c1a4849 name:ZooKeeperConnection Watcher:hd01:2181/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
2014-12-02 12:03:05,700 INFO org.apache.solr.common.cloud.ConnectionManager: Client is connected to ZooKeeper
2014-12-02 12:03:05,736 INFO org.apache.solr.common.cloud.ZkStateReader: Updating cluster state from ZooKeeper...
2014-12-02 12:03:06,694 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (1)
2014-12-02 12:03:06,754 INFO org.apache.solr.cloud.ZkController: Register node as live in ZooKeeper:/live_nodes/hd02:8983_solr
2014-12-02 12:03:06,760 INFO org.apache.solr.common.cloud.SolrZkClient: makePath: /live_nodes/hd02:8983_solr
2014-12-02 12:03:06,771 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (3)
2014-12-02 12:03:06,778 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (4)
2014-12-02 12:03:06,824 INFO org.apache.solr.servlet.SolrDispatchFilter: user.dir=/run/cloudera-scm-agent/process/984-solr-SOLR_SERVER
2014-12-02 12:03:06,824 INFO org.apache.solr.servlet.SolrDispatchFilter: SolrDispatchFilter.init() done
2014-12-02 12:03:25,745 INFO org.apache.solr.servlet.SolrDispatchFilter: [admin] webapp=null path=/admin/cores params={action=STATUS&wt=json} status=0 QTime=51

[#3 · Accepted solution]

Hello masfworld,

I've found my solution here, hope this'll help you too: http://community.cloudera.com/t5/Cloudera-Search-Apache-SolrCloud/Solr-Server-not-starting/m-p/4839#...