Running Custom Scripts through Ambari

[#1]

This article describes how to add a new custom script and execute the script through Ambari. In this article, I took a simple use case to install kerberos packages through custom scripts.

1) Below is the code to install kerberos packages (install_kerberos_package.py)

#!/usr/bin/env python

from resource_management import Script, Execute, format
from ambari_commons.os_check import OSCheck
from resource_management.core import shell
from resource_management.core.logger import Logger
class InstallKerberosPackage(Script):
  def actionexecute(self, env):
    config = Script.get_config()
    structured_output = {}
    cmd = self.get_install_cmd()
    Logger.info("Installing Kerberos Package")
    code, output = shell.call(cmd, sudo = True)
    if 0 == code:
      structured_output["install_kerberos_package"] = {"exit_code" : 0, "message": format("Packages installed successfully")}
    else:
      structured_output["install_kerberos_package"] = {"exit_code": code, "message": "Failed to install packages! {0}".format(str(output))}
    self.put_structured_out(structured_output)
  def get_install_cmd(self):
    if OSCheck.is_redhat_family():
      Logger.info("Installing kerberos package for the RedHat OS family");
      return ("/usr/bin/yum", "install", "-y", "krb5-server", "krb5-libs", "krb5-workstation")
    elif OSCheck.is_suse_family():
      Logger.info("Installing kerberos package for the SUSE OS family");
      return ('/usr/bin/zypper', '-n','install', 'krb5', 'krb5-server', 'krb5-client')
    elif OSCheck.is_ubuntu_family():
      Logger.info("Installing kerberos package for the Ubuntu OS family");
      return ('/usr/bin/apt-get', 'install', '-y', 'krb5-kdc', 'krb5-admin-server')
    else:
      raise Exception("Unsupported OS family: '{0}' ".format(OSCheck.get_os_family()))
if __name__ == "__main__":
  InstallKerberosPackage().execute() 

2) Save the file with install_kerberos_package.py and put it in the custom actions folder of ambari and change the permissions

cp install_kerberos_package.py /var/lib/ambari-server/resources/custom_actions/scripts/
chmod 755 /var/lib/ambari-server/resources/custom_actions/scripts/install_kerberos_package.py
chown root:root /var/lib/ambari-server/resources/custom_actions/scripts/install_kerberos_package.py

3) The next step is to add the definition for this action. Open the file system_action_definitions.xml located under "/var/lib/ambari-server/resources/custom_action_definitions/" folder using your favourite editor and add the below content alertDefinitions tag

<actionDefinition>
    <actionName>install_kerberos_package</actionName>
    <actionType>SYSTEM</actionType>
    <inputs></inputs>
    <targetService/>
    <targetComponent/>
    <defaultTimeout>60</defaultTimeout>
    <description>Install kerberos packages</description>
    <targetType>ALL</targetType>
    <permissions>HOST.ADD_DELETE_COMPONENTS, HOST.ADD_DELETE_HOSTS, SERVICE.ADD_DELETE_SERVICES</permissions>
  </actionDefinition>

Note: You can add comma separated inputs if you need any inputs for the script.

4) Now that the action is defined, restart ambari server for the changes to reflect.

ambari-server restart

5) Check if your action is listed by calling the Ambari REST API.

curl -u <username>:<password> "http://<ambari-host>:<ambari-port>/api/v1/actions"

6) Now you are ready to run the custom script which you have created.

curl -u <username>:<password> -X POST -H 'X-Requested-By:ambari' -d'{"RequestInfo":{"context":"Execute an action", "action" : "install_kerberos_package", "service_name" : "", "component_name":"", "hosts":"<comma-separated-hosts>"}}' http://<ambari-host>:<ambari-port>/api/v1/clusters/<cluster-name>/requests

You can check the output in the Ambari UI. Please check the screenshot(sample-output.png) for reference

[#2]

@Aditya Sirna

The mentioned command in the post is not working:

curl -u <username>:<password>-X POST -H 'X-Requested-By:ambari'-d'{"RequestInfo":{"context":"Execute an action", "action" : "install_kerberos_package", "service_name" : "", "component_name":"", "hosts":"<comma-separated-hosts>"}}' http://<ambari-host>:<ambari-port>/api/v1/clusters/<cluster-name>/requests

I am working in a Azure HDInsight Hadoop Cluster having 2 headnodes [primary and secondary] and 2 worker nodes and 1 edge node. I have configured the script and restarted ambari server to reflect the changes as you've mentioned but want to execute the script specific to primary head node.

By executing the command hostname -f, I got the FQDN of that host and execute the following command from edge node:

curl -u admin:<password> -X POST -H 'X-Requested-By:ambari' -d'{"RequestInfo":{"context":"Execute an action", "acton" : "validate_nfs_server_export_path", "service_name" : "", "component_name":"", "hosts":"hn0-kalyan.qjhddkgqet0efmcgr4zymppb4e.cx.internal.cloudapp.net"}}' http://headnodehost:8080/api/v1/clusters/cluster_name/requests

But when I saw the AMBARI UI, it shows executed on all the all nodes which it should not be.

[#3]

@Aditya Sirna

After going through the Ambari API documentation, what I found that the given curl command to execute the custom script action needs to be modified as I have tested and it's working fine.

curl -u <username>:<password> -X POST -H 'X-Requested-By:ambari' -d'{"RequestInfo":{"context":"Execute an action", "action" : "<action_name>"}, "Requests/resource_filters":[{"service_name" : "", "component_name":"", "hosts":"<comma_separated_host_names>"}]}' http://ambari_host:port/api/v1/clusters/cluster_name/requests

For specific components manageability:

curl -k -i -u <ambari_admin_user>:<ambari_admin_password> -H 'X-Requested-By: ambari' -X 
  POST -d '{"RequestInfo":
      {"query":"Hosts/host_name.in(<comma separated list of
       hosts to be added, if one node give only one
       name>)"},
     "Body":{"host_components":[{"HostRoles":{"component_name":"BIGSQL_WORKER"}}
       ]}}'
      http://<ambari-host>:
       <ambari-port>/api/v1/clusters/<cluster_name>/hosts?

curl -k -i -u <ambari_admin_user>:<ambari_admin_password> -H 'X-Requested-By: ambari' -X 
  PUT -d '{ "HostRoles": {"state":"INSTALLED" } }' 

http://<ambari-host>:<ambari-port>/api/v1/clusters/<cluster-name>/host_components?HostRoles/state=IN...

[#4]

Changes in system_action_definitions.xml is not picking up by ambari-server even after restart. Even I have kept the <actionName.py> file in the respective diretory. Could you please help me out?

[#5]

I have resolved the issue by placing ACTIVEAMBARIHOST with headnode FQDN name as from Edge Node it was not reflecting.

[#6]

Thanks for the article, it is very helpful.

I'd like to point out that in the article you included the note:

Note: You can add comma separated inputs if you need any inputs for the script.

That is great because I need to pass my script an input. I was able to create the custom action definition so that it expects and input (as you suggested was possible). Unfortunately I can't find anywhere that tells/shows how to pass the input. I have tried something like the following:

curl -u <username>:<password> -X POST -H 'X-Requested-By:ambari' -d'{"RequestInfo":{"context":"Execute my action", "action":"my_action"}, "Requests/resource_filters":[{"service_name":"", "component_name":"", "hosts":"<comma_separated_host_names>"}], "Requests/inputs":{"my_input":"input_value"}}' http://<ambari_host>:<port>/api/v1/clusters/<cluster_name>/requests

but that returns a 500 error saying:

An internal system exception occurred. Action my_action requires input 'my_input' that is not provided.

All my attempts so far have resulted in a 500 return code with the same message. So how am I supposed to pass the custom action its input via curl?

[#7]

@Keith Swanson,

You should pass your inputs inside parameters

curl -u <username>:<password> -X POST -H 'X-Requested-By:ambari' -d'{"RequestInfo":{"context":"Execute my action", "action":"my_action", "parameters" : {"my_input" : "value"}}, "Requests/resource_filters":[{"service_name":"", "component_name":"", "hosts":"<comma_separated_host_names>"}]' http://<ambari_host>:<port>/api/v1/clusters/<cluster_name>/requests

[#8]

@Aditya Sirna

Thank you for the quick response! That was exactly what I needed.