Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

Update YARN resource pools using cm api

SOLVED Go to solution

Update YARN resource pools using cm api

Rising Star

Hi Team,

 

Can anyone share if they have updated resource pool configurations in YARN using CM API, what end points were used if so. Thanks in advance.

2 ACCEPTED SOLUTIONS

Accepted Solutions

Re: Update YARN resource pools using cm api

Rising Star

Turns out, its a limitation as of now. Updates made to resource pools using CM API is known to break DRP UI. There are improvement tickets open internally with Cloudera to address that.

Re: Update YARN resource pools using cm api

Rising Star

Ways to change the pools via API today:

  1. Use the PUT call of http://$HOSTNAME:7180/api/v19/clusters/<cluster>/services/<yarn>/config, to change yarn_fs_scheduled_allocations, followed by a POST to refresh pools (http://$HOSTNAME:7180/api/v19/clusters/<cluster>/commands/poolsRefresh
    1. Pros: It does update the pools, as desired. It does NOT affect the web UI
    2. Cons: The JSON is complex and prone to typos. A typo could mess up all pools and cause issues on the cluster
3 REPLIES 3

Re: Update YARN resource pools using cm api

Rising Star

Specifically looking for :

 

Is this doable through the CM API?

 
Can I adjust the weights, memory, CPU of a Yarn resource pool?
Can I adjust create and delete pools?
Can I do the same allocations for Admission Control in Impala?

Re: Update YARN resource pools using cm api

Rising Star

Turns out, its a limitation as of now. Updates made to resource pools using CM API is known to break DRP UI. There are improvement tickets open internally with Cloudera to address that.

Re: Update YARN resource pools using cm api

Rising Star

Ways to change the pools via API today:

  1. Use the PUT call of http://$HOSTNAME:7180/api/v19/clusters/<cluster>/services/<yarn>/config, to change yarn_fs_scheduled_allocations, followed by a POST to refresh pools (http://$HOSTNAME:7180/api/v19/clusters/<cluster>/commands/poolsRefresh
    1. Pros: It does update the pools, as desired. It does NOT affect the web UI
    2. Cons: The JSON is complex and prone to typos. A typo could mess up all pools and cause issues on the cluster