Hi, One of the edge nodes in the cluster is generating alerts once or twice a week due to missed heartbeat from agent. INFO 2018-01-29 07:04:38,554 logger.py:71 - call returned (0, '') INFO 2018-01-29 07:06:04,226 logger.py:71 - call[['test', '-w', '/']] {'sudo': True, 'timeout': 5} INFO 2018-01-29 07:06:04,233 logger.py:71 - call returned (0, '') As you can see, there is no logging for 1.5 minutes and it is causing ambari alert for this edge node. How can i track if there was any connectivity issue between server and agent. Sometimes more than 1 heartbeat interval is missing. ... View more

Hi, I have changed my hadoop-env.sh and advanced log4j using ambari for making change to how hdfs-audit.log is rolled. I have added a new RollingFileAppender configuration in log4j as below, so that i can replace the DailyRollingFileAppender with this one. #Added below log4j.appender.RFAAUDIT=org.apache.log4j.RollingFileAppender log4j.appender.RFAAUDIT.File=${hadoop.log.dir}/hdfs-audit.log log4j.appender.RFAAUDIT.layout=org.apache.log4j.PatternLayout log4j.appender.RFAAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n log4j.appender.RFAAUDIT.MaxFileSize=100MB log4j.appender.RFAAUDIT.MaxBackupIndex=9 Now i am simply overwriting the advanced hadoop-env.sh with the new appender information Only replacing -Dhdfs.audit.logger=INFO,DRFAAUDIT with -Dhdfs.audit.logger=INFO,RFAAUDIT I have not touched the jobsummary logger -Dhadoop.mapreduce.jobsummary.logger=INFO,JSA But after restarting and monitoring the logs, i find that there is no issue with rolling of hdfs-audit.log. It is successfully rolling after 100MB. However my /var/log/hadoop-yarn/yarn/hadoop-mapreduce.jobsummary.log is not longer getting updated post this change. The log has stopped updating since this change. I don't know how the audit log settings have messed up the mapreduce jobsummary logging. ... View more