New Contributor
Posts: 1
Registered: ‎01-24-2014

Can we generate user logs in cluster?

[ Edited ]

I am facing memory issue on system when any map reduce job generate logs, can any one answer below questions

1. Can we generate userlogs in cluster? if yes how and what would be the configuration?
2. Can we delete user logs of running job? if yes, there would be any impact on running job?

Posts: 416
Topics: 51
Kudos: 86
Solutions: 49
Registered: ‎06-26-2013

Re: Can we generate user logs in cluster?

@cloudway I'm not sure I understand your request, but Mapreduce jobs automatically create their own logs and you can find them by browsing to your JobTracker's Web UI and clicking on the job itself.  On the job page, there will be a link to the logs.  You can also click down into any particular task of that job and see it's sdtout/stderr logs in the JT web UI.  By default, the JT keeps the logs around for about 24 hours and then archives them to a "job history" location.  They are eventually deleted from there too.


Does that answer your question?