YARN logs for applicationId - for specific period

[#1]

Hello - is there a way to get the YARN logs for an applicationId for a specific period ?

when i use command :

yarn logs application -applicationId <applicationId>  -log_files spark.log 

.. it is not giving me the older log files (eg. 2 days old log files)

any way for to get this log file, w/o having to goto the consolidated yarn resource manager log files

btw, the yarn logs retention is set to 30 days in yarn-site.xml

Another question :

whet s the option -> -log_files used for ? what are the options i can provide for this ?

[#2 · Accepted solution]

When you use "-log_files spark.log", you are basically asking for files of exact same name (spark.log). Probably the files older than 2 days are rotated to different names and hence not showing up.

Alternately, you can use "-log_files_pattern <Log File Pattern>" to fetch all files which match a pattern. So maybe something like this will fetch all old logs for you "-log_files_pattern spark*"

[#3]

thanks, that helps !