JobHistory UI is not showing the running jobs.

[#1]

I have a job running in the cluster, but I am unable to see that job through the JobHistory UI. I can only see the job if I execute the command "hadoop job -list" in the linux command prompt. I have observed that if I go to "ResourceManager UI" I see a running application, but I do not see any jobs of that running application through "JobHistory UI". In the ResourceManager UI, I have also observed that the latest application that I executed is associated with "ApplicationMaster' under "Tracking UI" field. Rest of the other Applications are associated with "History" under "Tracking UI" field. Is this the reason why I cannot see all the jobs under JobHistory UI for this application because it is associated with ApplicationMaster?.

[#2 · Accepted solution]

@Pradeep kumar When a application is currently running it will not be available in JobHistory UI until it is finished. As you correctly identified its tracking id will point to "Application Master". Once it is finished link will point to history server.

After the completion of a map reduce job, logs are written to hdfs under the directory specified by mapreduce.jobhistory.intermediate-done-dir. History server continuously scans the intermediate directory and pulls any new logs if available and copies those logs to the directory specified by mapreduce.jobhistory.done-dir

[#3]

@Ajay I have understood it now, but can you tell me how do I view the Current mappers running for an application? what links should I follow?

[#4]

@Pradeep kumar ResourceManager UI-> Application -> ApplicationMaster-> job link-> map/reducer link