Prevent Solr Cloud Logs from Filling Up a Node

[#1]

When one stands up SolrCloud/HDPSearch, every core/replica will be writing logs to /opt/lucidworks-hdpsearch/solr/server/logs. Files like 'solr-8983-console.log' will grow very quickly for a lightly used index, at the rate of several GBs per day. The reason is it outputs all INFO and DEBUG messages to this console log. Which is great for development, but a pain for production and operations. The challenge is there are quite a few log configuration files spread around the Solr directories, and those will not necessarily yield the desired result. It gets even more complicated for a SolrCloud, where one is dealing with a cluster of nodes.

There is, however, a very straightforward way to quickly address this issue:

1. Navigate to the admin console, like http://xxx-solr-1:8983/solr
2. Click on Logging -> Level
3. Select a root logger (click on the level value) and modify it to be WARN

The change is applied at runtime, no restart required.

[#2]

I noticed this change doesn't persist after restart, any pointers here?

[#3]

Hey Andrew, I'm having the same issue. Did you find any fixes for this? It's great it can be changed without a restart... but having to come back in and set this anytime Solr or the server itself is restarted isn't great. 🙂

EDIT: Help from Hortonworks support pointed out that in log4j.properties (either in Ambari if your Solr Cloud is managed there, or directly on server if not) you can adjust the following line from:

log4j.rootLogger=INFO, file, CONSOLE

Changed to:

log4j.rootLogger=WARN, file, CONSOLE

That will allow it to retain this change on Solr startup.