Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to rotate HDFS daemon log

Highlighted

How to rotate HDFS daemon log

New Contributor

Hi All, I have a lots of hdfs daemon logs for

1. hadoop-hdfs-datanode-<namenode host>.log

2. hadoop-hdfs-journalnode-<namenode host>.log

3. hadoop-hdfs-secondarynamenode-<namenode host>.log

4. hadoop-hdfs-namenode-<namenode host>.log

5. hadoop-hdfs-zkfc-<namenode host>.log

6. hadoop-hdfs-nfs3-<namenode host>.log

Can anyone tell how to rotate the above logs or where & how can I define the rotate policy for above logs?

2 REPLIES 2

Re: How to rotate HDFS daemon log

New Contributor

HI @Palash Dutta

By default, hadoop daemon logs should be enabled with the "RFA"(RollingFileAppender), which is configured with 250MB max size (hadoop.log.maxfilesize) for each file and 20 (hadoop.log.maxbackupindex) such backups can be kept for each daemon's logs.

So if you want to reduce the max size of each log file and/or reduce total number of max backup log files, you can change the above value in log4j.properties (Be sure to change the first log4j.properties included in the classpath).

Re: How to rotate HDFS daemon log

Contributor

Hi @Palash Dutta ,

I have attached HDFS log4j file, using that file you will be able to rotate the mentioned files on daily basis.
hdfs-log4j-drfa.txt
Update in Ambari UI > HDFS > Configs > Advanced > Advanced hdfs-log4j
Restarted is required for Affected components.

Don't have an account?
Coming from Hortonworks? Activate your account here