05-19-2015 07:57 AM
I have created a new user (user2) and when running any MapReduce jobs the JobHistoryServer portal (port 19888) does NOT show any!!!
I can see the job from witihin the ResourceManager Web UI (port 8088) and when I click on the 'History' link (under the Tracking UI column) I am getting the error: "No Found: job_xxxxxxxxxxxxx_xxxx).
This happening for the MapReduce jobs only!!!
When I run Spark Jobs (under the same userid, user2) I am able to see the 'History' logs!!!
An existing userid (user1) works fine!
It seems to me that the issue is permissions-related.
Here are the HDFS permissions of the userids:
[root@master ~]# hdfs dfs -ls /tmp/logs
Found 5 items
drwxrwxrwt - user1 hadoop 0 2015-04-20 08:48 /tmp/logs/user1
drwxrwxrwt - hdfs supergroup 0 2015-04-09 14:59 /tmp/logs/hdfs
drwxrwxrwx - user2 hadoop 0 2015-05-06 16:23 /tmp/logs/user2
drwxrwxrwt - root hadoop 0 2015-04-09 16:46 /tmp/logs/root
Also, here are the persmissions of the HDFS /user/history/done and /user/history/done_intermediate folders:
[root@master ~]# hdfs dfs -ls /user/history
Found 2 items
drwxrwx--- - mapred hadoop 0 2015-05-11 10:32 /user/history/done
drwxrwxrwt - mapred hadoop 0 2015-05-18 14:39 /user/history/done_intermediate
05-20-2015 11:11 AM
05-22-2015 06:59 AM
Thank you for your comment!
The issue has been resolved, it had to do with permissions!
I had to reset the mode of the /user/history/done folder /user/history/done_intermediate was good!