Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 12-25-2017 01:31 PM
Hi Team,
Is there any way to add new queue using Yarn rest api ? or Ambari configs.sh script ?
I tried below link but no luck -
https://community.hortonworks.com/questions/33578/api-to-manage-yarn-capacity-queue.html
http://<ambari_server>:8080/api/v1/views/CAPACITY-SCHEDULER/versions/1.0.0/instances/AUTO_CS_INSTANC... [root@node1 ~]# curl -u admin:admin -H "X-Requested-By:ambari" -iX PUT -d @cs.json http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic HTTP/1.1 100 Continue HTTP/1.1 400 Bad Request X-Frame-Options: DENY X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff Cache-Control: no-store Pragma: no-cache Set-Cookie: AMBARISESSIONID=6a3wvwuvrhhu3e5258edqhjt;Path=/;HttpOnly Expires: Thu, 01 Jan 1970 00:00:00 GMT User: admin Content-Type: text/plain Content-Length: 217 { "status" : 400, "message" : "org.apache.ambari.server.controller.spi.UnsupportedPropertyException: The properties [items] specified in the request or predicate are not supported for the resource type Cluster."
When i tried to enable Developer Tools for Ambari View->Yarn queue manager , I see below call -
http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic/configurations?type=capacity-scheduler&tag=ver... ======= { "href" : "http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic/configurations?type=capacity-scheduler&tag=version1514209101120", "items" : [ { "href" : "http://192.168.56.111:8080/api/v1/clusters/hdp_mosaic/configurations?type=capacity-scheduler&tag=version1514209101120", "tag" : "version1514209101120", "type" : "capacity-scheduler", "version" : 5, "Config" : { "cluster_name" : "hdp_mosaic", "stack_id" : "HDP-2.5" }, "properties" : { "yarn.scheduler.capacity.maximum-am-resource-percent" : "0.2", "yarn.scheduler.capacity.maximum-applications" : "10000", "yarn.scheduler.capacity.node-locality-delay" : "40", "yarn.scheduler.capacity.queue-mappings-override.enable" : "false", "yarn.scheduler.capacity.resource-calculator" : "org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator", "yarn.scheduler.capacity.root.MosaiqQueue.acl_administer_queue" : "*", "yarn.scheduler.capacity.root.MosaiqQueue.acl_submit_applications" : "*", "yarn.scheduler.capacity.root.MosaiqQueue.capacity" : "90", "yarn.scheduler.capacity.root.MosaiqQueue.maximum-capacity" : "90", "yarn.scheduler.capacity.root.MosaiqQueue.minimum-user-limit-percent" : "100", "yarn.scheduler.capacity.root.MosaiqQueue.ordering-policy" : "fifo", "yarn.scheduler.capacity.root.MosaiqQueue.state" : "RUNNING", "yarn.scheduler.capacity.root.MosaiqQueue.user-limit-factor" : "1", "yarn.scheduler.capacity.root.accessible-node-labels" : "*", "yarn.scheduler.capacity.root.acl_administer_queue" : "*", "yarn.scheduler.capacity.root.capacity" : "100", "yarn.scheduler.capacity.root.default.acl_submit_applications" : "*", "yarn.scheduler.capacity.root.default.capacity" : "10", "yarn.scheduler.capacity.root.default.maximum-capacity" : "100", "yarn.scheduler.capacity.root.default.state" : "RUNNING", "yarn.scheduler.capacity.root.default.user-limit-factor" : "1", "yarn.scheduler.capacity.root.queues" : "MosaiqQueue,default" } } ] }
Created 03-01-2018 06:22 AM
I was able to achieve this using ambari api updating service configs.
Below is the working command - I have added Queue name - "MaxiqQueue"
curl -u $ambari_user:$ambari_password -H 'X-Requested-By:admin' -X PUT "http://$ambari_server_host:8080/api/v1/clusters/$CLUSTER_NAME" -d '{
"Clusters": {
"desired_config": {
"type": "capacity-scheduler",
"tag": "version'$date'",
"properties": {
"yarn.scheduler.capacity.maximum-am-resource-percent" : "0.2",
"yarn.scheduler.capacity.maximum-applications" : "10000",
"yarn.scheduler.capacity.node-locality-delay" : "40",
"yarn.scheduler.capacity.queue-mappings-override.enable" : "false",
"yarn.scheduler.capacity.resource-calculator" : "org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator",
"yarn.scheduler.capacity.root.MaxiqQueue.acl_administer_queue" : "*",
"yarn.scheduler.capacity.root.MaxiqQueue.acl_submit_applications" : "*",
"yarn.scheduler.capacity.root.MaxiqQueue.capacity" : "90",
"yarn.scheduler.capacity.root.MaxiqQueue.maximum-capacity" : "90",
"yarn.scheduler.capacity.root.MaxiqQueue.minimum-user-limit-percent" : "100",
"yarn.scheduler.capacity.root.MaxiqQueue.ordering-policy" : "fifo",
"yarn.scheduler.capacity.root.MaxiqQueue.state" : "RUNNING",
"yarn.scheduler.capacity.root.MaxiqQueue.user-limit-factor" : "1",
"yarn.scheduler.capacity.root.accessible-node-labels" : "*",
"yarn.scheduler.capacity.root.acl_administer_queue" : "yarn",
"yarn.scheduler.capacity.root.capacity" : "100",
"yarn.scheduler.capacity.root.default.acl_administer_queue" : "yarn",
"yarn.scheduler.capacity.root.default.acl_submit_applications" : "yarn",
"yarn.scheduler.capacity.root.default.capacity" : "10",
"yarn.scheduler.capacity.root.default.maximum-capacity" : "100",
"yarn.scheduler.capacity.root.default.state" : "RUNNING",
"yarn.scheduler.capacity.root.default.user-limit-factor" : "1",
"yarn.scheduler.capacity.root.queues" : "MaxiqQueue,default"
}
}
}
}'
Created 03-16-2018 07:56 PM
@Veerendra Nath Jasthi As I am adding queues programmatically, I am hesitant to call API on views ... I am calling refresh queues api as follows, for which I need to manually do the save and refresh.
curl -u uname:pwd -H "X-Requested-By:ambari" -iX POST -d '{
"RequestInfo" : {
"command" : "REFRESHQUEUES",
"context" : "Refresh YARN Capacity Scheduler",
"operation_level" : {
"level": "HOST_COMPONENT",
"cluster_name": "<clustername>"
},
"parameters/forceRefreshConfigTags":"capacity-scheduler"
},
"Requests/resource_filters": [{
"service_name" : "YARN",
"component_name" : "RESOURCEMANAGER",
"hosts" : "<hostname>"
}]
}' http://<hostip>:<port>/api/v1/clusters/<clustername>/requests