Created 01-17-2016 07:25 PM
Hi,
I am facing lots of "bad dir status" issues on Nodemanagers while running huge job. This is caused by flooding the container log files with [DEBUG] messages (in detail the 'syslog' file is getting very large).
How can I modify the loglevel of the container specific log to reduce the size of the syslog file ?
E.g.:
$ ls -alh ./container_e12_1453036276967_0004_01_000004: total 7.4G drwxr-s--- 2 siad hadoop 4.0K Jan 17 18:39 . drwxr-s--- 10 siad hadoop 4.0K Jan 17 20:21 .. -rw-r----- 1 siad hadoop 222 Jan 17 18:47 stderr -rw-r----- 1 siad hadoop 0 Jan 17 18:39 stdout -rw-r----- 1 siad hadoop 7.4G Jan 17 18:47 syslog
In /etc/hadoop I cannot find any config setting including the value 'debug', so where does this DEBUG output come from ?
Thanks, Gerd
Created 02-04-2016 05:06 PM
Seems like for that particular job the log setting got overwritten by a local container-log4j.properties, which included DEBUG loglevel.
Created 02-04-2016 05:06 PM
@All, sorry for any inconvenience 😉
Created 02-04-2016 05:09 PM
@Gerd Koenig good job, re-accepted to give you credit