Skip to main content
Prav
Prav
Gold ⭐️
January 8, 2019
Solved

Update YARN resource pools using cm api

Hi Team,

 

Can anyone share if they have updated resource pool configurations in YARN using CM API, what end points were used if so. Thanks in advance.

    Prav
    PravAnswer
    Gold ⭐️
    March 5, 2019

    Ways to change the pools via API today:

    1. Use the PUT call of http://$HOSTNAME:7180/api/v19/clusters/<cluster>/services/<yarn>/config, to change yarn_fs_scheduled_allocations, followed by a POST to refresh pools (http://$HOSTNAME:7180/api/v19/clusters/<cluster>/commands/poolsRefresh) 
      1. Pros: It does update the pools, as desired. It does NOT affect the web UI
      2. Cons: The JSON is complex and prone to typos. A typo could mess up all pools and cause issues on the cluster
    View in thread

    3 replies

    Prav
    PravAuthor
    Gold ⭐️
    January 9, 2019

    Specifically looking for :

     

    Is this doable through the CM API?

     
    Can I adjust the weights, memory, CPU of a Yarn resource pool?
    Can I adjust create and delete pools?
    Can I do the same allocations for Admission Control in Impala?
    Translate
    Prav
    PravAuthor
    Gold ⭐️
    January 16, 2019

    Turns out, its a limitation as of now. Updates made to resource pools using CM API is known to break DRP UI. There are improvement tickets open internally with Cloudera to address that.

    Translate
    Prav
    PravAuthorAnswer
    Gold ⭐️
    March 5, 2019

    Ways to change the pools via API today:

    1. Use the PUT call of http://$HOSTNAME:7180/api/v19/clusters/<cluster>/services/<yarn>/config, to change yarn_fs_scheduled_allocations, followed by a POST to refresh pools (http://$HOSTNAME:7180/api/v19/clusters/<cluster>/commands/poolsRefresh) 
      1. Pros: It does update the pools, as desired. It does NOT affect the web UI
      2. Cons: The JSON is complex and prone to typos. A typo could mess up all pools and cause issues on the cluster
    Translate
    Terms & ConditionsAccessibility statement