Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

JobHistory UI is not showing the running jobs.

avatar
Expert Contributor

I have a job running in the cluster, but I am unable to see that job through the JobHistory UI. I can only see the job if I execute the command "hadoop job -list" in the linux command prompt. I have observed that if I go to "ResourceManager UI" I see a running application, but I do not see any jobs of that running application through "JobHistory UI". In the ResourceManager UI, I have also observed that the latest application that I executed is associated with "ApplicationMaster' under "Tracking UI" field. Rest of the other Applications are associated with "History" under "Tracking UI" field. Is this the reason why I cannot see all the jobs under JobHistory UI for this application because it is associated with ApplicationMaster?.

1 ACCEPTED SOLUTION

avatar
Expert Contributor

@Pradeep kumar When a application is currently running it will not be available in JobHistory UI until it is finished. As you correctly identified its tracking id will point to "Application Master". Once it is finished link will point to history server.

After the completion of a map reduce job, logs are written to hdfs under the directory specified by mapreduce.jobhistory.intermediate-done-dir. History server continuously scans the intermediate directory and pulls any new logs if available and copies those logs to the directory specified by mapreduce.jobhistory.done-dir

View solution in original post

3 REPLIES 3

avatar
Expert Contributor

@Pradeep kumar When a application is currently running it will not be available in JobHistory UI until it is finished. As you correctly identified its tracking id will point to "Application Master". Once it is finished link will point to history server.

After the completion of a map reduce job, logs are written to hdfs under the directory specified by mapreduce.jobhistory.intermediate-done-dir. History server continuously scans the intermediate directory and pulls any new logs if available and copies those logs to the directory specified by mapreduce.jobhistory.done-dir

avatar
Expert Contributor

@Ajay I have understood it now, but can you tell me how do I view the Current mappers running for an application? what links should I follow?

avatar
Expert Contributor

@Pradeep kumar ResourceManager UI-> Application -> ApplicationMaster-> job link-> map/reducer link