question Re: YARN logs for applicationId - for specific period in Archives of Support Questions (Read Only)

YARN logs for applicationId - for specific period

karan_alang1 — Fri, 15 Dec 2017 05:27:30 GMT

Hello - is there a way to get the YARN logs for an applicationId for a specific period ?

when i use command :

yarn logs application -applicationId <applicationId>  -log_files spark.log

.. it is not giving me the older log files (eg. 2 days old log files)

any way for to get this log file, w/o having to goto the consolidated yarn resource manager log files

btw, the yarn logs retention is set to 30 days in yarn-site.xml

Another question :

whet s the option -> -log_files used for ? what are the options i can provide for this ?

Re: YARN logs for applicationId - for specific period

gsaha — Fri, 15 Dec 2017 05:53:55 GMT

When you use "-log_files spark.log", you are basically asking for files of exact same name (spark.log). Probably the files older than 2 days are rotated to different names and hence not showing up.

Alternately, you can use "-log_files_pattern <Log File Pattern>" to fetch all files which match a pattern. So maybe something like this will fetch all old logs for you "-log_files_pattern spark*"

Re: YARN logs for applicationId - for specific period

karan_alang1 — Fri, 15 Dec 2017 07:46:28 GMT

thanks, that helps !