Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

YARN logs for applicationId - for specific period

avatar
Expert Contributor

Hello - is there a way to get the YARN logs for an applicationId for a specific period ?

when i use command :

yarn logs application -applicationId <applicationId>  -log_files spark.log 

.. it is not giving me the older log files (eg. 2 days old log files)

any way for to get this log file, w/o having to goto the consolidated yarn resource manager log files

btw, the yarn logs retention is set to 30 days in yarn-site.xml

Another question :

whet s the option -> -log_files used for ? what are the options i can provide for this ?

1 ACCEPTED SOLUTION

avatar
Expert Contributor

When you use "-log_files spark.log", you are basically asking for files of exact same name (spark.log). Probably the files older than 2 days are rotated to different names and hence not showing up.

Alternately, you can use "-log_files_pattern <Log File Pattern>" to fetch all files which match a pattern. So maybe something like this will fetch all old logs for you "-log_files_pattern spark*"

View solution in original post

2 REPLIES 2

avatar
Expert Contributor

When you use "-log_files spark.log", you are basically asking for files of exact same name (spark.log). Probably the files older than 2 days are rotated to different names and hence not showing up.

Alternately, you can use "-log_files_pattern <Log File Pattern>" to fetch all files which match a pattern. So maybe something like this will fetch all old logs for you "-log_files_pattern spark*"

avatar
Expert Contributor

thanks, that helps !