Member since
05-15-2015
8
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3715 | 05-18-2015 03:30 PM |
08-24-2015
03:53 PM
Hi - yarn.scheduler.maximum-allocation-vcores controls the maximum vcores that any submitted job can request. yarn.nodemanager.resource.cpu-vcores, on the other hand, controls how many vcores can be scheduled on a particular NodeManager instance. So yarn.nodemanager.resource.cpu-vcores can vary from host to host (NodeManager to NodeManager), while yarn.scheduler.maximum-allocation-vcores is a global property of the scheduler. Regards, Mark
... View more
05-18-2015
03:30 PM
1 Kudo
Hi - What version of Cloudera Manager are you running? Cloudera Manager 5.0, unfortunately, had a bug that prevented the YARN Job History Server from picking up changes to the configured value for the YARN remote application log directory. This was fixed in CM 5.1. If you are running CM 5.1+, the problem could have to do with the ownership or permissions of the remote application log directory (the HDFS directory where the logs are stored). By default, this is "/tmp/logs", but it can be configured to a different value in CM (YARN > Configuration > search for "remote app log dir"). This HDFS directory (and subdirectories) needs to be readable by group hadoop. Please check the ownership/permissions using "hdfs dfs -ls -R /tmp" from the command line (replacing "/tmp" with the value of YARN's remote app log dir, if it has been set to a different value). The group ownership should be "hadoop," and the permissions should allow group read access. If this is not the case, you can use "hdfs dfs -chown -R mapred:hadoop to set the group ownership recursively. Regards, Mark
... View more
05-15-2015
04:27 PM
Hi, If you click on the "Applications" link near the top of the YARN service page, you'll be taken to a page with information about YARN jobs. Clicking on a job ID link on that page will display a summary page for the job. From there, you can drill down into individual map and reduce tasks, and view the associated logs. Regards, Mark
... View more