Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can someone help me to find the complete hdp 3.0 yarn GPU process of setup scheduling and isolation

Highlighted

Can someone help me to find the complete hdp 3.0 yarn GPU process of setup scheduling and isolation

New Contributor

I struggle to find the complete guidance how to isolate and schedule jobs application GPUs in hdp3.0 ? With hdp3.0 GPU is as a native asset, so are only gpu isolation and scheduling the only improvements ?

1 REPLY 1

Re: Can someone help me to find the complete hdp 3.0 yarn GPU process of setup scheduling and isolation

New Contributor

For now (3.1.1/3.2.0) the capacity.CapacityScheduler is broken by a hardcoded enum containing only vCores and RAM parameters.
You just have to switch your scheduler class to org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler
You also want to replace "capacity" by "Fair" in the line
yarn.scheduler.fair.resource-calculator=org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

Your GPUs will not be visible on yarn ui2 but will still be on the NodeManagers, and most importantly, will be allocated properly.
It was a mess to find out indeed.