Reply
Highlighted
Contributor
Posts: 35
Registered: ‎07-06-2018
Accepted Solution

Update YARN resource pools using cm api

[ Edited ]

Hi Team,

 

Can anyone share if they have updated resource pool configurations in YARN using CM API, what end points were used if so. Thanks in advance.

Contributor
Posts: 35
Registered: ‎07-06-2018

Re: Update YARN resource pools using cm api

Specifically looking for :

 

Is this doable through the CM API?

 
Can I adjust the weights, memory, CPU of a Yarn resource pool?
Can I adjust create and delete pools?
Can I do the same allocations for Admission Control in Impala?
Contributor
Posts: 35
Registered: ‎07-06-2018

Re: Update YARN resource pools using cm api

Turns out, its a limitation as of now. Updates made to resource pools using CM API is known to break DRP UI. There are improvement tickets open internally with Cloudera to address that.

Contributor
Posts: 35
Registered: ‎07-06-2018

Re: Update YARN resource pools using cm api

Ways to change the pools via API today:

  1. Use the PUT call of http://$HOSTNAME:7180/api/v19/clusters/<cluster>/services/<yarn>/config, to change yarn_fs_scheduled_allocations, followed by a POST to refresh pools (http://$HOSTNAME:7180/api/v19/clusters/<cluster>/commands/poolsRefresh
    1. Pros: It does update the pools, as desired. It does NOT affect the web UI
    2. Cons: The JSON is complex and prone to typos. A typo could mess up all pools and cause issues on the cluster
Announcements