Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Custom script based alert for a non-Hadoop process running on single or multiple nodes

Highlighted

Custom script based alert for a non-Hadoop process running on single or multiple nodes

Contributor

Please help me configure a custom script based alert for a non-Hadoop process running on single or multiple nodes

I’ve refereed to these articles already.

https://github.com/apache/ambari/blob/branch-2.1/ambari-server/docs/api/v1/alert-definitions.md https://cwiki.apache.org/confluence/display/AMBARI/Customizing+the+Alert+Template

Script: XYZ-process-chk.sh

#!/bin/sh
PROCESS_CHK=`code`
if [ code ]
then
	echo "process is NOT rounning"
fi

JSON file:

{
  "AlertDefinition" : {
    "cluster_name" : "NAME",
    "alert.hasHostName" : "host1", "host2",   # I may need this property
    "component_name" : "XYZ",
    "description" : null,
    "enabled" : true,
    "help_url" : null,
    "ignore_host" : false,
    "interval" : 1,
    "label" : "XYZ Process",
    "name" : "XYZ_process",
    "repeat_tolerance" : 1,
    "repeat_tolerance_enabled" : false,
    "scope" : "HOST",
    "service_name" : "XYZ",
  "source" : {
    "path" : "/var/lib/ambari-server/resources/host_scripts/XYZ-process-chk.sh",
    "type" : "SCRIPT"
   }
}
3 REPLIES 3

Re: Custom script based alert for a non-Hadoop process running on single or multiple nodes

Super Collaborator

Currently, the alerts framework doesn't allow running scripts which are not Python-based. Therefore, you'd first need to convert your shell script into a Python file similar to alerts which ship with the product.

Once you have a running Python script alert, then you need to figure out where to target it. It you want it running on every host, then you'd target it similar to how the agent host alerts work. It would be distributed to each host on your cluster. See: https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/alerts.json#L123-L164

If you want to target specific hosts, then you'd need to "install" a new service in Ambari which lets Ambari know where to distribute the alerts. Let's say you wanted to have an alert which checks to see if the foo-process is running. This process is a part of the Foo service which is installed on your master hosts only. You'd need to create a new "Service" in Ambari using the stack extension mechanism and then install the Foo Process components on the right hosts using Ambari.

Some administrators go with option #2 because they want Ambari to manage the non-Hadoop service. However, it's a bit more complicated. You can use option #1 and simply have your alert check for the existance of the process on the path. If it exists, you know to check for it running. If it doesn't exist, then you can return the SKIPPED alert state.

Re: Custom script based alert for a non-Hadoop process running on single or multiple nodes

Contributor

Thanks for the quick response. I might have to go with opt #2 as the process might be running on a single or multiple nodes but not all. I will keep you posted.

Re: Custom script based alert for a non-Hadoop process running on single or multiple nodes

Super Collaborator

Sure - as I mentioned, an option that some people take is to have it run on all hosts, but to return the SKIPPED state if the script determines this host isn't of interest. Some ways to do this:

- The alert can check for the existence of the process on the file system. If it's not there, it knows that this host shouldn't be included in the alerts.

- You can add a property to any configuration, including cluster-env, which lists the hosts which should be checked. The scripts have access to all configurations, so they can see if the current host is in the list.