Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 01-17-2016 07:25 PM
Hi,
I am facing lots of "bad dir status" issues on Nodemanagers while running huge job. This is caused by flooding the container log files with [DEBUG] messages (in detail the 'syslog' file is getting very large).
How can I modify the loglevel of the container specific log to reduce the size of the syslog file ?
E.g.:
$ ls -alh ./container_e12_1453036276967_0004_01_000004: total 7.4G drwxr-s--- 2 siad hadoop 4.0K Jan 17 18:39 . drwxr-s--- 10 siad hadoop 4.0K Jan 17 20:21 .. -rw-r----- 1 siad hadoop 222 Jan 17 18:47 stderr -rw-r----- 1 siad hadoop 0 Jan 17 18:39 stdout -rw-r----- 1 siad hadoop 7.4G Jan 17 18:47 syslog
In /etc/hadoop I cannot find any config setting including the value 'debug', so where does this DEBUG output come from ?
Thanks, Gerd
Created 02-04-2016 05:06 PM
Seems like for that particular job the log setting got overwritten by a local container-log4j.properties, which included DEBUG loglevel.
Created 02-04-2016 05:06 PM
@All, sorry for any inconvenience 😉
Created 02-04-2016 05:09 PM
@Gerd Koenig good job, re-accepted to give you credit