Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

JobHistory UI is not showing the running jobs.

Solved Go to solution

JobHistory UI is not showing the running jobs.

Expert Contributor

I have a job running in the cluster, but I am unable to see that job through the JobHistory UI. I can only see the job if I execute the command "hadoop job -list" in the linux command prompt. I have observed that if I go to "ResourceManager UI" I see a running application, but I do not see any jobs of that running application through "JobHistory UI". In the ResourceManager UI, I have also observed that the latest application that I executed is associated with "ApplicationMaster' under "Tracking UI" field. Rest of the other Applications are associated with "History" under "Tracking UI" field. Is this the reason why I cannot see all the jobs under JobHistory UI for this application because it is associated with ApplicationMaster?.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: JobHistory UI is not showing the running jobs.

Rising Star

@Pradeep kumar When a application is currently running it will not be available in JobHistory UI until it is finished. As you correctly identified its tracking id will point to "Application Master". Once it is finished link will point to history server.

After the completion of a map reduce job, logs are written to hdfs under the directory specified by mapreduce.jobhistory.intermediate-done-dir. History server continuously scans the intermediate directory and pulls any new logs if available and copies those logs to the directory specified by mapreduce.jobhistory.done-dir

3 REPLIES 3
Highlighted

Re: JobHistory UI is not showing the running jobs.

Rising Star

@Pradeep kumar When a application is currently running it will not be available in JobHistory UI until it is finished. As you correctly identified its tracking id will point to "Application Master". Once it is finished link will point to history server.

After the completion of a map reduce job, logs are written to hdfs under the directory specified by mapreduce.jobhistory.intermediate-done-dir. History server continuously scans the intermediate directory and pulls any new logs if available and copies those logs to the directory specified by mapreduce.jobhistory.done-dir

Re: JobHistory UI is not showing the running jobs.

Expert Contributor

@Ajay I have understood it now, but can you tell me how do I view the Current mappers running for an application? what links should I follow?

Re: JobHistory UI is not showing the running jobs.

Rising Star

@Pradeep kumar ResourceManager UI-> Application -> ApplicationMaster-> job link-> map/reducer link

Don't have an account?
Coming from Hortonworks? Activate your account here