Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Update YARN resource pools using cm api

avatar
Contributor

Hi Team,

 

Can anyone share if they have updated resource pool configurations in YARN using CM API, what end points were used if so. Thanks in advance.

2 ACCEPTED SOLUTIONS

avatar
Contributor

Turns out, its a limitation as of now. Updates made to resource pools using CM API is known to break DRP UI. There are improvement tickets open internally with Cloudera to address that.

View solution in original post

avatar
Contributor

Ways to change the pools via API today:

  1. Use the PUT call of http://$HOSTNAME:7180/api/v19/clusters/<cluster>/services/<yarn>/config, to change yarn_fs_scheduled_allocations, followed by a POST to refresh pools (http://$HOSTNAME:7180/api/v19/clusters/<cluster>/commands/poolsRefresh
    1. Pros: It does update the pools, as desired. It does NOT affect the web UI
    2. Cons: The JSON is complex and prone to typos. A typo could mess up all pools and cause issues on the cluster

View solution in original post

3 REPLIES 3

avatar
Contributor

Specifically looking for :

 

Is this doable through the CM API?

 
Can I adjust the weights, memory, CPU of a Yarn resource pool?
Can I adjust create and delete pools?
Can I do the same allocations for Admission Control in Impala?

avatar
Contributor

Turns out, its a limitation as of now. Updates made to resource pools using CM API is known to break DRP UI. There are improvement tickets open internally with Cloudera to address that.

avatar
Contributor

Ways to change the pools via API today:

  1. Use the PUT call of http://$HOSTNAME:7180/api/v19/clusters/<cluster>/services/<yarn>/config, to change yarn_fs_scheduled_allocations, followed by a POST to refresh pools (http://$HOSTNAME:7180/api/v19/clusters/<cluster>/commands/poolsRefresh
    1. Pros: It does update the pools, as desired. It does NOT affect the web UI
    2. Cons: The JSON is complex and prone to typos. A typo could mess up all pools and cause issues on the cluster