I am experimenting with LogSearch and I have noticed that the history collection in ambari-infra has only one shard and on replication. I tried to change that but everytime I restart logsearch, the history collection is reset to the default. Is there a place I can make my change permanent? like having more shards and replication factors?
In tech preview, event history is not yet supported, its only contains some log filter data (for 3.0 that will be moved to zookeeper).
History shard and replication number can be set in /etc/ambari-logsearch-portal/conf/logsearch.properties file.
For now it set in a python file (that fills logsearch.properties https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/resources/common-services/LO... which will generate the logsearch.properties file.
If you are using ambari-2.5 you can add custom properties for logsearch-properties config type and override "logsearch.collection.history.replication.factor" and "logsearch.collection.history.numshards" as well.
In case you are using ambari-2.4, it will be a bit harder because that is hard coded in a jinja template file (
https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/resources/common-services/LO..., so you will need to update that file, which is a bit hacky, maybe you can also try out that solution what i suggested for 2.5
Other than that, I think you can update shard replication numbers through solr api as well, after logsearch is started
Thank you @oszabo for the detailed answer. I am using Ambari 2.4.2 and I did try your suggestions for the jinja template but it does not seem to be working. Would you happen to know the location in the logsearch code where the collections are created? this way I can back track the config to the right config parameters.
In 2.4, that will be a bit harder to change, as i see that was hard coded in the java code as well:
Then, I suggest to create the collection manually before starting logsearch application.