I have setup YARN Fair-scheduler in Ambari (HDP 188.8.131.52-78) for "Default" queue itself. So far, I haven't added any new queues.
Now, I want to run a simple job against the queue and when I submit the job, the application state is in "ACCEPTED" state forever. I get the below message in YARN logs:
The additional information is given below. Please help me in fixing this issue at the earliest.
For "default" queue, the below parameters are set through "fair-scheduler.xml".
Also, no jobs are currently running, apart from the one that I have launched.
Given below is the screenshot, which confirms that the maximum AM resource percent is greater than 0.1
Please take a look at:
What type of FairScheduler are you using:
What is the weight of the default queue you are submitting your apps to?
From my perspictive you are limiting your default queue to use at minimum 1024MB 0vCores and at maximum 8196MB 0vCores. In both cases no cores are set - when you try to run a job it requires to run with 1024MB memory and 1vCores - it then fails to allocate the 1vCore due to 0vCore min/max restriction and it sends 'exceeds maximum AM resources allowed'
That's why I think the issue is with the core utilization and not with memory.
In your screenshot the
Is set to 8192 mb, 0vcore
And your job requires at least 1vcore as seen in the Diagnostics section.
Please try increasing the vcore size in <queueMaxResourcesDefault> and try to run the job again.