Support Questions

Find answers, ask questions, and share your expertise

About the scheduler-capacity prop to launch many applications to the cluster

avatar

Hello,

 

My Queue's Absolute used capacity is at 91.66667 % when i launch one application with spark-submit. What causes this ?

 

Thanks for clarifying this as i get an error if i submit 2 applications at the same time to my cluster getting the second app stuck in ACCEPTED with the message : 

 

INFO Client:
         client token: N/A
         diagnostics: [Wed May 06 08:38:12 +0000 2020]  Application is Activated, waiting for resources to be assigned for AM. Details : AM Partition = <DEFAULT_PARTITION> ; Partition Resource = <memory:98304, vCores:28> ; Queue's Absolute capacity = 100.0 % ; Queue's Absolute used capacity = 91.66667 % ; Queue's Absolute max capacity = 100.0 % ; Queue's capacity (absolute resource) = <memory:98304, vCores:28> ; Queue's used capacity (absolute resource) = <memory:90112, vCores:4> ; Queue's max capacity (absolute resource) = <memory:98304, vCores:28> ;
         ApplicationMaster host: N/A
         ApplicationMaster RPC port: -1
         queue: tab
         start time: 1588754291472
         final status: UNDEFINED
         tracking URL: http://hadoop-master:8088/proxy/application_1588667960453_0011/
         user: tab

Which property of capacity-scheduler.xml affects this behaviour ?

1 REPLY 1

avatar
Contributor

Can you check the userlimit of the queue and the max AM resource percentage ? 
RM UI -> Scheduler -> Expand your queue( take screenshot and attach to this case)