Zookeeper log file not rotated

[#1]

Hi,

We are using the below hadoop version. Most of the time the zookeeper log file zookeeper.out grows so large around GBs and doesn't get rotated. Let us know the possible solutions.

Hadoop 2.7.1.2.3.2.0-2950 Subversion git@github.com:hortonworks/hadoop.git -r 5cc60e0003e33aa98205f18bccaeaf36cb193c1c Compiled by jenkins on 2015-09-30T18:08Z Compiled with protoc 2.5.0 From source with checksum 69a3bf8c667267c2c252a54fbbf23d This command was run using /usr/hdp/2.3.2.0-2950/hadoop/lib/hadoop-common-2.7.1.2.3.2.0-2950.jar

Regards,

Venkadesh S

[#2]

The zookeeper "*.out" log is controlled by the "appender.CONSOLE" . Console appenders does not have rolling feature available in it. So you have following options:

1. Try using the OS utilities like "logrotate" to rotate the out log of zokeeper.

2. Change the log level of CONSOLE appender to ERROR/WARNING

log4j.rootLogger=ERROR, CONSOLE

3. Other wise switch to "ROLLINGFILE" instead of CONSOLE file appender.

log4j.rootLogger=INFO, ROLLINGFILE
log4j.appender.ROLLINGFILE.File=${zookeeper.log.dir}/zookeeper.log 

How ever it will be good to know why the zookeeper logs are growing so much ? Do you see a repeated warning/error? Or you see only INFO messages are filling the .out log ?

[#3]

@Joy There are not much error or warning. Most of it is INFO is getting printed as below.

INFO [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version),

[#4 · Accepted solution]

In that case the best option will be to switch to "ROLLINGFILE" as it has the capability to roll

log4j.rootLogger=INFO, ROLLINGFILE
log4j.appender.ROLLINGFILE.File=${zookeeper.log.dir}/zookeeper.log 
log4j.appender.ROLLINGFILE.MaxFileSize=10MB
log4j.appender.ROLLINGFILE.MaxBackupIndex=10