Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

How to add new yarn queue using Rest API / Ambari configs.sh script

avatar
Expert Contributor

Hi Team,

Is there any way to add new queue using Yarn rest api ? or Ambari configs.sh script ?

I tried below link but no luck -

https://community.hortonworks.com/questions/33578/api-to-manage-yarn-capacity-queue.html

http://<ambari_server>:8080/api/v1/views/CAPACITY-SCHEDULER/versions/1.0.0/instances/AUTO_CS_INSTANC...


[root@node1 ~]#  curl -u admin:admin -H "X-Requested-By:ambari" -iX PUT -d @cs.json http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic
HTTP/1.1 100 Continue


HTTP/1.1 400 Bad Request
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Cache-Control: no-store
Pragma: no-cache
Set-Cookie: AMBARISESSIONID=6a3wvwuvrhhu3e5258edqhjt;Path=/;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
User: admin
Content-Type: text/plain
Content-Length: 217


{
  "status" : 400,
  "message" : "org.apache.ambari.server.controller.spi.UnsupportedPropertyException: The properties [items] specified in the request or predicate are not supported for the resource type Cluster."

When i tried to enable Developer Tools for Ambari View->Yarn queue manager , I see below call -

http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic/configurations?type=capacity-scheduler&tag=ver...

=======

{
  "href" : "http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic/configurations?type=capacity-scheduler&tag=version1514209101120",
  "items" : [
    {
      "href" : "http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic/configurations?type=capacity-scheduler&tag=version1514209101120",
      "tag" : "version1514209101120",
      "type" : "capacity-scheduler",
      "version" : 5,
      "Config" : {
        "cluster_name" : "hdp_mosaic",
        "stack_id" : "HDP-2.5"
      },
      "properties" : {
        "yarn.scheduler.capacity.maximum-am-resource-percent" : "0.2",
        "yarn.scheduler.capacity.maximum-applications" : "10000",
        "yarn.scheduler.capacity.node-locality-delay" : "40",
        "yarn.scheduler.capacity.queue-mappings-override.enable" : "false",
        "yarn.scheduler.capacity.resource-calculator" : "org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator",
        "yarn.scheduler.capacity.root.MosaiqQueue.acl_administer_queue" : "*",
        "yarn.scheduler.capacity.root.MosaiqQueue.acl_submit_applications" : "*",
        "yarn.scheduler.capacity.root.MosaiqQueue.capacity" : "90",
        "yarn.scheduler.capacity.root.MosaiqQueue.maximum-capacity" : "90",
        "yarn.scheduler.capacity.root.MosaiqQueue.minimum-user-limit-percent" : "100",
        "yarn.scheduler.capacity.root.MosaiqQueue.ordering-policy" : "fifo",
        "yarn.scheduler.capacity.root.MosaiqQueue.state" : "RUNNING",
        "yarn.scheduler.capacity.root.MosaiqQueue.user-limit-factor" : "1",
        "yarn.scheduler.capacity.root.accessible-node-labels" : "*",
        "yarn.scheduler.capacity.root.acl_administer_queue" : "*",
        "yarn.scheduler.capacity.root.capacity" : "100",
        "yarn.scheduler.capacity.root.default.acl_submit_applications" : "*",
        "yarn.scheduler.capacity.root.default.capacity" : "10",
        "yarn.scheduler.capacity.root.default.maximum-capacity" : "100",
        "yarn.scheduler.capacity.root.default.state" : "RUNNING",
        "yarn.scheduler.capacity.root.default.user-limit-factor" : "1",
        "yarn.scheduler.capacity.root.queues" : "MosaiqQueue,default"
      }
    }
  ]
}
1 ACCEPTED SOLUTION

avatar
Expert Contributor
@pjoseph

I was able to achieve this using ambari api updating service configs.

Below is the working command - I have added Queue name - "MaxiqQueue"

 curl -u $ambari_user:$ambari_password -H 'X-Requested-By:admin' -X PUT "http://$ambari_server_host:8080/api/v1/clusters/$CLUSTER_NAME" -d '{
  "Clusters": {
    "desired_config": {
      "type": "capacity-scheduler",
      "tag": "version'$date'",
      "properties": {
            "yarn.scheduler.capacity.maximum-am-resource-percent" : "0.2",
            "yarn.scheduler.capacity.maximum-applications" : "10000",
            "yarn.scheduler.capacity.node-locality-delay" : "40",
            "yarn.scheduler.capacity.queue-mappings-override.enable" : "false",
            "yarn.scheduler.capacity.resource-calculator" : "org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator",
            "yarn.scheduler.capacity.root.MaxiqQueue.acl_administer_queue" : "*",
            "yarn.scheduler.capacity.root.MaxiqQueue.acl_submit_applications" : "*",
            "yarn.scheduler.capacity.root.MaxiqQueue.capacity" : "90",
            "yarn.scheduler.capacity.root.MaxiqQueue.maximum-capacity" : "90",
            "yarn.scheduler.capacity.root.MaxiqQueue.minimum-user-limit-percent" : "100",
            "yarn.scheduler.capacity.root.MaxiqQueue.ordering-policy" : "fifo",
            "yarn.scheduler.capacity.root.MaxiqQueue.state" : "RUNNING",
            "yarn.scheduler.capacity.root.MaxiqQueue.user-limit-factor" : "1",
            "yarn.scheduler.capacity.root.accessible-node-labels" : "*",
            "yarn.scheduler.capacity.root.acl_administer_queue" : "yarn",
            "yarn.scheduler.capacity.root.capacity" : "100",
            "yarn.scheduler.capacity.root.default.acl_administer_queue" : "yarn",
            "yarn.scheduler.capacity.root.default.acl_submit_applications" : "yarn",
            "yarn.scheduler.capacity.root.default.capacity" : "10",
            "yarn.scheduler.capacity.root.default.maximum-capacity" : "100",
            "yarn.scheduler.capacity.root.default.state" : "RUNNING",
            "yarn.scheduler.capacity.root.default.user-limit-factor" : "1",
            "yarn.scheduler.capacity.root.queues" : "MaxiqQueue,default"
      }
    }
  }
}'

View solution in original post

10 REPLIES 10

avatar
New Member

@Veerendra Nath Jasthi As I am adding queues programmatically, I am hesitant to call API on views ... I am calling refresh queues api as follows, for which I need to manually do the save and refresh.

curl -u uname:pwd -H "X-Requested-By:ambari" -iX POST -d '{
  "RequestInfo" : {
    "command" : "REFRESHQUEUES",
    "context" : "Refresh YARN Capacity Scheduler",
    "operation_level" : {
        "level": "HOST_COMPONENT",
        "cluster_name": "<clustername>"
    },
    "parameters/forceRefreshConfigTags":"capacity-scheduler"
  },
  "Requests/resource_filters": [{
    "service_name" : "YARN",
    "component_name" : "RESOURCEMANAGER",
    "hosts" : "<hostname>"
  }]
}' http://<hostip>:<port>/api/v1/clusters/<clustername>/requests