Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Mapreduce logs missing


Mapreduce logs missing

Super Guru

I have yarn.log-aggregation-enable set to true and yarn.nodemanager.remote-app-log-dir set to /app-logs

Once the job finished I don't see the directory /app-logs in hdfs. I am unable to view the job logs through resource manager. The yarn.log-aggregation.retain-seconds is set to high number. Any ideas?


Re: Mapreduce logs missing

Rising Star

@Sunile Manjee Check the NM and RM logs and grep for "aggrega". There are often times errors related to aggregation. You may need to create the /app-logs dir in hdfs and ensure proper ownership. I would start by making sure the aggregation thread is running and seen in the log. If it is, then it is a filesystem level issue.

Don't have an account?
Coming from Hortonworks? Activate your account here