YARN ResourceManager REST API throwing erro?

[#1]

I was trying change the configuration of Capacity-scheduler on the fly using rest-api call. Below is the configuration look like:

curl -v -u admin:admin -H "Content-Type: application/json" -H "X-Requested-By:ambari" -X PUT http://localhost:8080/api/v1/views/CAPACITY-SCHEDULER/versions/1.0.0/instances/AUTO_CS_INSTANCE/reso... --data '{ "Clusters": { "desired_config": [ { "type": "capacity-scheduler", "tag": "version14534007568115", "service_config_version_note": "To test", "properties": { "yarn.scheduler.capacity.maximum-am-resource-percent": 0.2, "yarn.scheduler.capacity.maximum-applications": 10000, "yarn.scheduler.capacity.node-locality-delay": 40, "yarn.scheduler.capacity.resource-calculator": "org.apache.hadoop.yarn.util.resource.DominantResourceCalculator", "yarn.scheduler.capacity.queue-mappings-override.enable": false, "yarn.scheduler.capacity.root.acl_administer_queue": "*", "yarn.scheduler.capacity.root.capacity": 100, "yarn.scheduler.capacity.root.queues": "Hive", "yarn.scheduler.capacity.root.accessible-node-labels": "*", "yarn.scheduler.capacity.root.Hive.acl_submit_applications": "*", "yarn.scheduler.capacity.root.Hive.maximum-capacity": 100, "yarn.scheduler.capacity.root.Hive.user-limit-factor": 4, "yarn.scheduler.capacity.root.Hive.state": "RUNNING", "yarn.scheduler.capacity.root.Hive.capacity": 100 } } ] } }'

getting error like :

About to connect() to localhost port 8080 (#0) * Trying localhost... Connection refused * couldn't connect to host * Closing connection #0 curl: (7) couldn't connect to host

In our env. we dont have kerberos enabled.

[#2]

@Veerendra Nath Jasthi, is your Ambari Server running on localhost?

[#3]

Note, you are hitting an Ambari REST API and not RM REST API.

[#4]

@Gour Saha

https://community.hortonworks.com/content/supportkb/49421/how-to-administer-capacity-scheduler-view-...

the above link says that we should hit ambari to change the configurations on the fly.

No I do have ip address but cant share that so why I named it as a localhost whenever I am trying to call I will have ip in place of localhost that ambari server is running...

[#5]

Hi Saha I was able to run this but ending up error...

RESTAPI :

curl -u admin:admin -H "X-Requested-By: ambari" -X PUT $ambari_server/api/v1/clusters/engsndbx -d '[{"Clusters":{ > "desired_config":[{ > "type":"capacity-scheduler", > "tag":"version1519845495539", > "properties":{ > "yarn.scheduler.capacity.maximum-am-resource-percent":"0.4", > "yarn.scheduler.capacity.maximum-applications":"10000", > "yarn.scheduler.capacity.node-locality-delay":"40", > "yarn.scheduler.capacity.resource-calculator": "org.apache.hadoop.yarn.util.resource.DominantResourceCalculator", > "yarn.scheduler.capacity.queue-mappings-override.enable":"false", > "yarn.scheduler.capacity.root.acl_administer_queue": "*", > "yarn.scheduler.capacity.root.capacity":"100", > "yarn.scheduler.capacity.root.queues": "Hive", > "yarn.scheduler.capacity.root.accessible-node-labels": "*", > "yarn.scheduler.capacity.root.Hive.acl_submit_applications": "*", > "yarn.scheduler.capacity.root.Hive.maximum-capacity":"100", > "yarn.scheduler.capacity.root.Hive.user-limit-factor":"4", > "yarn.scheduler.capacity.root.Hive.state": "RUNNING", > "yarn.scheduler.capacity.root.Hive.capacity":"100"}, > "service_config_version_note":"New config version"}]}}]'

from above rest api I would like to update the "yarn.scheduler.capacity.root.Hive.user-limit-factor":"4", but ending upwith error :

{ "status" : 500, "message" : "org.apache.ambari.server.controller.spi.SystemException: An internal system exception occurred: Configuration with tag 'version1519845495539' exists for 'capacity-scheduler'"

any suggestions ?

[#6]

You need to use a new tag, not one of the existing ones. Typically it is the "version" keyword followed by current timestamp.

If you don't absolutely need to use the REST API or you don't want to deal with the version tag, you should use configs.sh/configs.py (the sh is not supported in some older Ambari versions).

Sample get call is -

/var/lib/ambari-server/resources/scripts/configs.py -a get -l <ambari_server_host> -n <cluster_name> -c capacity-scheduler -f /tmp/cs.json

Sample output in /tmp/cs.json is -

{
  "properties": {
    "yarn.scheduler.capacity.maximum-am-resource-percent": "0.4", 
    "yarn.scheduler.capacity.maximum-applications": "10000", 
    "yarn.scheduler.capacity.node-locality-delay": "40", 
    "yarn.scheduler.capacity.resource-calculator": "org.apache.hadoop.yarn.util.resource.DominantResourceCalculator", 
    "yarn.scheduler.capacity.queue-mappings-override.enable": "false", 
    "yarn.scheduler.capacity.root.acl_administer_queue": "*", 
    "yarn.scheduler.capacity.root.capacity": "100", 
    "yarn.scheduler.capacity.root.queues": "Hive", 
    "yarn.scheduler.capacity.root.accessible-node-labels": "*", 
    "yarn.scheduler.capacity.root.Hive.acl_submit_applications": "*", 
    "yarn.scheduler.capacity.root.Hive.maximum-capacity": "100", 
    "yarn.scheduler.capacity.root.Hive.user-limit-factor": "4", 
    "yarn.scheduler.capacity.root.Hive.state": "RUNNING", 
    "yarn.scheduler.capacity.root.Hive.capacity": "100"
  }
}

Help is -

/var/lib/ambari-server/resources/scripts/configs.py -h

To perform the change you want to make, edit the file /tmp/cs.json with your desired change (update value of yarn.scheduler.capacity.root.Hive.user-limit-factor in your case) then use the "-a set" option with the same file. Sample cmd provided below -

/var/lib/ambari-server/resources/scripts/configs.py -a set -l <ambari_server_host> -n <cluster_name>-c capacity-scheduler -f /tmp/cs1.json

Note, you need to refresh queues to make this change take effect. You can do it by running rmadmin via command line -

yarn rmadmin -refreshQueues

Or, use the Ambari REST API -

curl -u admin:admin -H 'Content-Type:application/json' -H 'X-Requested-By:ambari' -iX PUT -d '{"save": "true"}' http://<ambari-server>:8080/api/v1/views/CAPACITY-SCHEDULER/versions/1.0.0/instances/<view_instance_...

You do not need to restart RM for capacity scheduler changes. However, if you make changes to other configs like yarn-site via configs.py you need to restart RM. You can do so by using Ambari REST APIs as shown below.

Stop RM -

curl -u admin:admin -H "X-Requested-By:ambari" -iX PUT -d '{"ServiceComponentInfo":{"state":"INSTALLED"}}' http://<ambari-server>:8080/api/v1/clusters/<cluster-name>/services/YARN/components/RESOURCEMANAGER

Start RM -

curl -u admin:admin -H "X-Requested-By:ambari" -iX PUT -d '{"ServiceComponentInfo":{"state":"STARTED"}}' http://<ambari-server>:8080/api/v1/clusters/<cluster-name>/services/YARN/components/RESOURCEMANAGER