Support Questions
Find answers, ask questions, and share your expertise

hadoop-mapreduce.jobsummary.log not updating after change in log4j settings in hadoop-env.sh

Explorer

Hi,

I have changed my hadoop-env.sh and advanced log4j using ambari for making change to how hdfs-audit.log is rolled. I have added a new RollingFileAppender configuration in log4j as below, so that i can replace the DailyRollingFileAppender with this one.

#Added below

log4j.appender.RFAAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.RFAAUDIT.File=${hadoop.log.dir}/hdfs-audit.log
log4j.appender.RFAAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.RFAAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
log4j.appender.RFAAUDIT.MaxFileSize=100MB
log4j.appender.RFAAUDIT.MaxBackupIndex=9

Now i am simply overwriting the advanced hadoop-env.sh with the new appender information

Only replacing

-Dhdfs.audit.logger=INFO,DRFAAUDIT with -Dhdfs.audit.logger=INFO,RFAAUDIT

I have not touched the jobsummary logger

-Dhadoop.mapreduce.jobsummary.logger=INFO,JSA

But after restarting and monitoring the logs, i find that there is no issue with rolling of hdfs-audit.log. It is successfully rolling after 100MB. However my /var/log/hadoop-yarn/yarn/hadoop-mapreduce.jobsummary.log is not longer getting updated post this change. The log has stopped updating since this change. I don't know how the audit log settings have messed up the mapreduce jobsummary logging.

1 REPLY 1

Explorer

Restarting the hdfs, yarn and mapreduce after some configuration change(not related to the post) did the trick.