Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

YARN logs for applicationId - for specific period

Solved Go to solution

YARN logs for applicationId - for specific period

Expert Contributor

Hello - is there a way to get the YARN logs for an applicationId for a specific period ?

when i use command :

yarn logs application -applicationId <applicationId>  -log_files spark.log 

.. it is not giving me the older log files (eg. 2 days old log files)

any way for to get this log file, w/o having to goto the consolidated yarn resource manager log files

btw, the yarn logs retention is set to 30 days in yarn-site.xml

Another question :

whet s the option -> -log_files used for ? what are the options i can provide for this ?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: YARN logs for applicationId - for specific period

Rising Star

When you use "-log_files spark.log", you are basically asking for files of exact same name (spark.log). Probably the files older than 2 days are rotated to different names and hence not showing up.

Alternately, you can use "-log_files_pattern <Log File Pattern>" to fetch all files which match a pattern. So maybe something like this will fetch all old logs for you "-log_files_pattern spark*"

2 REPLIES 2

Re: YARN logs for applicationId - for specific period

Rising Star

When you use "-log_files spark.log", you are basically asking for files of exact same name (spark.log). Probably the files older than 2 days are rotated to different names and hence not showing up.

Alternately, you can use "-log_files_pattern <Log File Pattern>" to fetch all files which match a pattern. So maybe something like this will fetch all old logs for you "-log_files_pattern spark*"

Highlighted

Re: YARN logs for applicationId - for specific period

Expert Contributor

thanks, that helps !