Suppose I have cluster consisting of 1 high configuration master node on which are installed all the master components I need and two medium configuration machines which only have the components of DataNode and NodeManager. Now, from what I could tell, there is no "Alert" among the list of alerts that would notify me if my cluster CPU(By this I refer to only the datanodes) usage is above a certain percentage, so that it can send me an alert whenever it does go above it. And Why Ambari is not providing such type of "Alert"?
By default in Ambari there is no predefined alert for monitoring Specifically CPU used by DataNode. But ambari provides you an option to create and register your own custom alert.
For example, I have written an article to register a custom alert in ambari: (Which will find the disk usage for different mount points and will trigger the alert according to the threshold defined.
Similarly you can also write your own script to calculate the DataNode CPU usage and then register the alert using "alerts.json" file as described in the above article.
You should be able to create the "Aggregate" type alert as well.
AGGREGATE alerts aggregate the alert status as a percentage of the alert instances affected. For example, the "Percent DataNodes Available" alert aggregates the DataNode Process alert. The threshold units are “%”.
However registering the custom alert process will be same as mentioned in my previous doc.
Hi Sir how can i find out more about writing custom alerts specifically what all sort of functions and other things are available in the ambari libraries that your custom script imports (for example : mount, OsFamilyFuncImpl, OsFamilyImpl,logger)
Most of the Ambari Python scripts you can find inside the "ambari-commons" (like OsFamilyFuncImpl, OsFamilyImpl,logger)
The author describes his alert script as alerting for allocated space in specified mount points. It is clear (for me) that mountPoints in the alert script is just referring to, indeed, a filesystem mount point.