Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Ambari REST API to restart all services

avatar
Contributor

Is there a simple cURL request to restart all services on a cluster (or preferably, just the ones with stale configurations)? I tried to follow the answer at this similarly-named post but the calls only stopped the services, and the calls the start them up again did not have any effect. If there is a way to simply restart the services that would be ideal.

Note: this entire process is running in a bash script, so I would prefer answers where the entire restart process can be easily automated (i.e. don't require me to use the Ambari UI or require arduous parsing of a curl JSON response).

Thanks for any and all help!

[1] https://community.hortonworks.com/questions/29439/ambari-api-to-restart-all-the-services-with-stale....

1 ACCEPTED SOLUTION

avatar
Expert Contributor

@Zach Kirsch :The problem in the script could be that the wait between stopping all services and starting it is not enough. The start immediately after stop would result in something of the sort :

{ "status" : 500, "message" : "org.apache.ambari.server.controller.spi.SystemException: An internal system exception occurred: Invalid transition for servicecomponenthost, clusterName=cl1, clusterId=2, serviceName=HDFS, componentName=SECONDARY_NAMENODE, hostname=nat-r6-dtxs-ambari-hosts-4-4.openstacklocal, currentState=STOPPING, newDesiredState=STARTED" }

Instead what you could do is parse the response of the call to put services to INSTALLED state and check that it is completed.

Code here (Assuming you have the ambari.props set up as in https://community.hortonworks.com/questions/29439/ambari-api-to-restart-all-the-services-with-stale....

curl -u $AMBARI_ADMIN_USER:$AMBARI_ADMIN_PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context": "put services into STOPPED STATE"},"Body":{"ServiceInfo": {"state" : "INSTALLED"}}}' "$URL" > /tmp/response.txt
newURL=`grep -o '"href" : [^, }]*' /tmp/response.txt | sed 's/^.*: //' | tr -d '"'`
echo newURL=$newURL
request_status=""
while [ "$request_status" != "COMPLETED" ];
do
        curl -u $AMBARI_ADMIN_USER:$AMBARI_ADMIN_PASSWORD -i -X GET "$newURL" > /tmp/new_response.txt
        request_status=`grep -o '"request_status" : [^, }]*' /tmp/new_response.txt | sed 's/^.*: //' | tr -d '"'`
        echo $request_status
done
curl -u $AMBARI_ADMIN_USER:$AMBARI_ADMIN_PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context": "put services into STARTED state"},"Body":{"ServiceInfo": {"state" : "STARTED"}}}' "$URL"

NOTE : This will fail if the services are all already in stopped state or if the stop of services fails (You will need to check in the while loop if "$request_status" = "FAILED", abort the script) These scripts give you bare minimum to get things to work. Extra checks needs to be added to make them fault tolerant (esp to timing issues).

View solution in original post

8 REPLIES 8

avatar
Rising Star
@Zach Kirsch

Check this article from @Kuldeep Kulkarni which has a bunch of scripts that would suit your requirement

https://community.hortonworks.com/articles/34747/ambari-admin-utility.html

./ambari-admin.sh restart refresh is the one

The source code of the above script as mentioned in the article is available at: https://github.com/crazyadmins/useful-scripts/blob/master/ambari/ambari-admin.sh

-Vivek

avatar
Master Guru

Thanks @vsharma 🙂

avatar
Contributor

The script only stops the services, doesn't start them back up. See the attached image. I can also attach the output of curl for the starting/stopping from the script if that is helpful. The curl requests to stop the services seem to work perfectly, but the calls to start up the services don't even have a body (Content-Length:0). capture1.png

avatar
Expert Contributor
@Zach Kirsch

Below is the link that would help you with all the api call that you can use to stop/start/delete/install any service:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=41812517

Here is an example to start the service:

curl -u admin:$PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Start HDFS via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' http://`hostname -f`:8080/api/v1/clusters/$CLUSTER_NAME/services/HDFS

avatar
Expert Contributor

@Zach Kirsch :The problem in the script could be that the wait between stopping all services and starting it is not enough. The start immediately after stop would result in something of the sort :

{ "status" : 500, "message" : "org.apache.ambari.server.controller.spi.SystemException: An internal system exception occurred: Invalid transition for servicecomponenthost, clusterName=cl1, clusterId=2, serviceName=HDFS, componentName=SECONDARY_NAMENODE, hostname=nat-r6-dtxs-ambari-hosts-4-4.openstacklocal, currentState=STOPPING, newDesiredState=STARTED" }

Instead what you could do is parse the response of the call to put services to INSTALLED state and check that it is completed.

Code here (Assuming you have the ambari.props set up as in https://community.hortonworks.com/questions/29439/ambari-api-to-restart-all-the-services-with-stale....

curl -u $AMBARI_ADMIN_USER:$AMBARI_ADMIN_PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context": "put services into STOPPED STATE"},"Body":{"ServiceInfo": {"state" : "INSTALLED"}}}' "$URL" > /tmp/response.txt
newURL=`grep -o '"href" : [^, }]*' /tmp/response.txt | sed 's/^.*: //' | tr -d '"'`
echo newURL=$newURL
request_status=""
while [ "$request_status" != "COMPLETED" ];
do
        curl -u $AMBARI_ADMIN_USER:$AMBARI_ADMIN_PASSWORD -i -X GET "$newURL" > /tmp/new_response.txt
        request_status=`grep -o '"request_status" : [^, }]*' /tmp/new_response.txt | sed 's/^.*: //' | tr -d '"'`
        echo $request_status
done
curl -u $AMBARI_ADMIN_USER:$AMBARI_ADMIN_PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context": "put services into STARTED state"},"Body":{"ServiceInfo": {"state" : "STARTED"}}}' "$URL"

NOTE : This will fail if the services are all already in stopped state or if the stop of services fails (You will need to check in the while loop if "$request_status" = "FAILED", abort the script) These scripts give you bare minimum to get things to work. Extra checks needs to be added to make them fault tolerant (esp to timing issues).

avatar
Contributor

Thanks for the reply - I have tried something similar to this, and tried not sending the "start" call until I saw the "stop" process had completely in the Ambari UI. The weird thing is that the response to the start call isn't an error message like you had predicted - it's just an empty response. It doesn't matter if I don't wait at all or wait 5 minutes, the response is empty. Is there any api call to "restart" so I don't have to manage the timing of stopping, waiting, starting?

EDIT: It looks like from here that clients can't be put into any state other than "installed". I'm going to try to use the Restart API as defined here.

$ curl --retry 5 --fail -u admin:$PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Starting SPARK_CLIENT"}, "HostRoles": {"state": "STARTED"}}' http://headnodehost:8080/api/v1/clusters/sparkucigraph39/hosts/wn3-sparku.cyzc0onq2zqudhbbi3tes4j3sd...
HTTP/1.1 200 OK
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
User: sparkadm
Set-Cookie: AMBARISESSIONID=13hxfu6ocnm4oumibnfjle633;Path=/;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/plain
Content-Length: 0
Server: Jetty(8.1.17.v20150415)

For comparison, here is the "stop" command:

$ curl --retry 5 --fail -u admin:$PASSWORD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo": {"context" :"Stopping SPARK_CLIENT"}, "HostRoles": {"state": "INSTALLED"}}' http://headnodehost:8080/api/v1/clusters/sparkucigraph39/hosts/wn3-sparku.cyzc0onq2zqudhbbi3tes4j3sd...
HTTP/1.1 202 Accepted
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
User: sparkadm
Set-Cookie: AMBARISESSIONID=4aelvrdeyd571rpff061lfifd;Path=/;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/plain
Vary: Accept-Encoding, User-Agent
Content-Length: 148
Server: Jetty(8.1.17.v20150415)


{
  "href" : "http://headnodehost:8080/api/v1/clusters/sparkucigraph39/requests/87",
  "Requests" : {
    "id" : 87,
    "status" : "Accepted"
  }
}

avatar
New Contributor

RESTART ALL SERVICES in a cluster

curl -u admin:$PASSWORD -i -H 'X-Requested-By: ambari' -X POST -d '{"RequestInfo": {"command": "RESTART","context": "Restart all services","operation_level": "host_component"},"Requests/resource_filters": [{"hosts_predicate": "HostRoles/cluster_name=$CLUSTER_NAME"}]}' http://headNodeHost:8080/api/v1/clusters/cl1/requests

avatar
Contributor

This worked for me. Upvoted