Support Questions

Find answers, ask questions, and share your expertise

How to add new yarn queue using Rest API / Ambari configs.sh script

avatar
Expert Contributor

Hi Team,

Is there any way to add new queue using Yarn rest api ? or Ambari configs.sh script ?

I tried below link but no luck -

https://community.hortonworks.com/questions/33578/api-to-manage-yarn-capacity-queue.html

http://<ambari_server>:8080/api/v1/views/CAPACITY-SCHEDULER/versions/1.0.0/instances/AUTO_CS_INSTANC...


[root@node1 ~]#  curl -u admin:admin -H "X-Requested-By:ambari" -iX PUT -d @cs.json http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic
HTTP/1.1 100 Continue


HTTP/1.1 400 Bad Request
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Cache-Control: no-store
Pragma: no-cache
Set-Cookie: AMBARISESSIONID=6a3wvwuvrhhu3e5258edqhjt;Path=/;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
User: admin
Content-Type: text/plain
Content-Length: 217


{
  "status" : 400,
  "message" : "org.apache.ambari.server.controller.spi.UnsupportedPropertyException: The properties [items] specified in the request or predicate are not supported for the resource type Cluster."

When i tried to enable Developer Tools for Ambari View->Yarn queue manager , I see below call -

http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic/configurations?type=capacity-scheduler&tag=ver...

=======

{
  "href" : "http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic/configurations?type=capacity-scheduler&tag=version1514209101120",
  "items" : [
    {
      "href" : "http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic/configurations?type=capacity-scheduler&tag=version1514209101120",
      "tag" : "version1514209101120",
      "type" : "capacity-scheduler",
      "version" : 5,
      "Config" : {
        "cluster_name" : "hdp_mosaic",
        "stack_id" : "HDP-2.5"
      },
      "properties" : {
        "yarn.scheduler.capacity.maximum-am-resource-percent" : "0.2",
        "yarn.scheduler.capacity.maximum-applications" : "10000",
        "yarn.scheduler.capacity.node-locality-delay" : "40",
        "yarn.scheduler.capacity.queue-mappings-override.enable" : "false",
        "yarn.scheduler.capacity.resource-calculator" : "org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator",
        "yarn.scheduler.capacity.root.MosaiqQueue.acl_administer_queue" : "*",
        "yarn.scheduler.capacity.root.MosaiqQueue.acl_submit_applications" : "*",
        "yarn.scheduler.capacity.root.MosaiqQueue.capacity" : "90",
        "yarn.scheduler.capacity.root.MosaiqQueue.maximum-capacity" : "90",
        "yarn.scheduler.capacity.root.MosaiqQueue.minimum-user-limit-percent" : "100",
        "yarn.scheduler.capacity.root.MosaiqQueue.ordering-policy" : "fifo",
        "yarn.scheduler.capacity.root.MosaiqQueue.state" : "RUNNING",
        "yarn.scheduler.capacity.root.MosaiqQueue.user-limit-factor" : "1",
        "yarn.scheduler.capacity.root.accessible-node-labels" : "*",
        "yarn.scheduler.capacity.root.acl_administer_queue" : "*",
        "yarn.scheduler.capacity.root.capacity" : "100",
        "yarn.scheduler.capacity.root.default.acl_submit_applications" : "*",
        "yarn.scheduler.capacity.root.default.capacity" : "10",
        "yarn.scheduler.capacity.root.default.maximum-capacity" : "100",
        "yarn.scheduler.capacity.root.default.state" : "RUNNING",
        "yarn.scheduler.capacity.root.default.user-limit-factor" : "1",
        "yarn.scheduler.capacity.root.queues" : "MosaiqQueue,default"
      }
    }
  ]
}
1 ACCEPTED SOLUTION

avatar
Expert Contributor
@pjoseph

I was able to achieve this using ambari api updating service configs.

Below is the working command - I have added Queue name - "MaxiqQueue"

 curl -u $ambari_user:$ambari_password -H 'X-Requested-By:admin' -X PUT "http://$ambari_server_host:8080/api/v1/clusters/$CLUSTER_NAME" -d '{
  "Clusters": {
    "desired_config": {
      "type": "capacity-scheduler",
      "tag": "version'$date'",
      "properties": {
            "yarn.scheduler.capacity.maximum-am-resource-percent" : "0.2",
            "yarn.scheduler.capacity.maximum-applications" : "10000",
            "yarn.scheduler.capacity.node-locality-delay" : "40",
            "yarn.scheduler.capacity.queue-mappings-override.enable" : "false",
            "yarn.scheduler.capacity.resource-calculator" : "org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator",
            "yarn.scheduler.capacity.root.MaxiqQueue.acl_administer_queue" : "*",
            "yarn.scheduler.capacity.root.MaxiqQueue.acl_submit_applications" : "*",
            "yarn.scheduler.capacity.root.MaxiqQueue.capacity" : "90",
            "yarn.scheduler.capacity.root.MaxiqQueue.maximum-capacity" : "90",
            "yarn.scheduler.capacity.root.MaxiqQueue.minimum-user-limit-percent" : "100",
            "yarn.scheduler.capacity.root.MaxiqQueue.ordering-policy" : "fifo",
            "yarn.scheduler.capacity.root.MaxiqQueue.state" : "RUNNING",
            "yarn.scheduler.capacity.root.MaxiqQueue.user-limit-factor" : "1",
            "yarn.scheduler.capacity.root.accessible-node-labels" : "*",
            "yarn.scheduler.capacity.root.acl_administer_queue" : "yarn",
            "yarn.scheduler.capacity.root.capacity" : "100",
            "yarn.scheduler.capacity.root.default.acl_administer_queue" : "yarn",
            "yarn.scheduler.capacity.root.default.acl_submit_applications" : "yarn",
            "yarn.scheduler.capacity.root.default.capacity" : "10",
            "yarn.scheduler.capacity.root.default.maximum-capacity" : "100",
            "yarn.scheduler.capacity.root.default.state" : "RUNNING",
            "yarn.scheduler.capacity.root.default.user-limit-factor" : "1",
            "yarn.scheduler.capacity.root.queues" : "MaxiqQueue,default"
      }
    }
  }
}'

View solution in original post

10 REPLIES 10

avatar

@Veerendra Nath Jasthi As I am adding queues programmatically, I am hesitant to call API on views ... I am calling refresh queues api as follows, for which I need to manually do the save and refresh.

curl -u uname:pwd -H "X-Requested-By:ambari" -iX POST -d '{
  "RequestInfo" : {
    "command" : "REFRESHQUEUES",
    "context" : "Refresh YARN Capacity Scheduler",
    "operation_level" : {
        "level": "HOST_COMPONENT",
        "cluster_name": "<clustername>"
    },
    "parameters/forceRefreshConfigTags":"capacity-scheduler"
  },
  "Requests/resource_filters": [{
    "service_name" : "YARN",
    "component_name" : "RESOURCEMANAGER",
    "hosts" : "<hostname>"
  }]
}' http://<hostip>:<port>/api/v1/clusters/<clustername>/requests