When you use "-log_files spark.log", you are basically asking for files of exact same name (spark.log). Probably the files older than 2 days are rotated to different names and hence not showing up. Alternately, you can use "-log_files_pattern <Log File Pattern>" to fetch all files which match a pattern. So maybe something like this will fetch all old logs for you "-log_files_pattern spark*"
... View more
