Support Questions

Find answers, ask questions, and share your expertise

How to rotate HDFS daemon log

Hi All, I have a lots of hdfs daemon logs for

1. hadoop-hdfs-datanode-<namenode host>.log

2. hadoop-hdfs-journalnode-<namenode host>.log

3. hadoop-hdfs-secondarynamenode-<namenode host>.log

4. hadoop-hdfs-namenode-<namenode host>.log

5. hadoop-hdfs-zkfc-<namenode host>.log

6. hadoop-hdfs-nfs3-<namenode host>.log

Can anyone tell how to rotate the above logs or where & how can I define the rotate policy for above logs?



HI @Palash Dutta

By default, hadoop daemon logs should be enabled with the "RFA"(RollingFileAppender), which is configured with 250MB max size (hadoop.log.maxfilesize) for each file and 20 (hadoop.log.maxbackupindex) such backups can be kept for each daemon's logs.

So if you want to reduce the max size of each log file and/or reduce total number of max backup log files, you can change the above value in (Be sure to change the first included in the classpath).

Rising Star

Hi @Palash Dutta ,

I have attached HDFS log4j file, using that file you will be able to rotate the mentioned files on daily basis.
Update in Ambari UI > HDFS > Configs > Advanced > Advanced hdfs-log4j
Restarted is required for Affected components.