I am using HDP-3.1 and YARN 3.1.1. The YARN application type is TEZ.
The hive mapreduce job used to work fine. But no mapduce job can run after a complete restart of the HDP. The MR job just hanged and I had to kill them. The job diagnosis show that
Application is added to the scheduler and is not yet activated. Skipping AM assignment as cluster resource is empty. Details : AM Partition = <DEFAULT_PARTITION>; AM Resource Request = <memory:4096, vCores:1>; Queue Resource Limit for AM = <memory:0, vCores:0>; User AM Resource Limit of the queue = <memory:0, vCores:0>; Queue AM Resource Usage = <memory:0, vCores:0>;
I changed the configuration of yarn.scheduler.capacity.maximum-am-resource-percent from 0.2 to 0.5 but still got the same result.
Another issue is that I do not know how to restart YARN. From Ambari UI, I used the menu button YARN - Actions - Restart All, and then clicked the "CONFIRM RESTART ALL" button to confirm. But nothing happens.
Created 08-26-2019 01:35 PM
My mistake. For some reason, the reboot of HDP did not restart YARN NodeManagers and I missed it from the Ambari UI. After I manually started each NodeManager, the MR job runs without an issue.
Created 08-26-2019 01:35 PM
My mistake. For some reason, the reboot of HDP did not restart YARN NodeManagers and I missed it from the Ambari UI. After I manually started each NodeManager, the MR job runs without an issue.