Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hello. How can I add customized Alert in Ambari for top 5 resource consuming jobs. The resources could be YARN Memory, Heap Memory or even Resource Manager Heap.

Hello. How can I add customized Alert in Ambari for top 5 resource consuming jobs. The resources could be YARN Memory, Heap Memory or even Resource Manager Heap.

New Contributor
 
5 REPLIES 5

Re: Hello. How can I add customized Alert in Ambari for top 5 resource consuming jobs. The resources could be YARN Memory, Heap Memory or even Resource Manager Heap.

Super Mentor

@Abhishek Vadnerkar

You can write your own logic using the Ambari Script Based Alert Dispatcher feature and then you should be able to get those custom alerts.

For example you can refer to this article to see an example of registering a custom alert in ambari: https://community.hortonworks.com/articles/38149/how-to-create-and-register-custom-ambari-alerts.htm...

.

For more details on this feature please refer to: https://cwiki.apache.org/confluence/display/AMBARI/Creating+a+Script-based+Alert+Dispatcher

.

Re: Hello. How can I add customized Alert in Ambari for top 5 resource consuming jobs. The resources could be YARN Memory, Heap Memory or even Resource Manager Heap.

New Contributor

Thanks for reply Jay SenSharma. So essentially, there is no way to make it happen on the Ambari UI by customizing the Widgets on Dashboard? Scripting is the only way?

Thanks

Abhishek

Re: Hello. How can I add customized Alert in Ambari for top 5 resource consuming jobs. The resources could be YARN Memory, Heap Memory or even Resource Manager Heap.

Super Mentor

@Abhishek Vadnerkar

In Ambari defined default alerts i do not see any alert definitions which can currently list "top 5 resource consuming jobs" Because this is really a very custom requirement so you will have to write your own logic to get the list of those top5 resources.

Although ambari has some of the memory utilization related alerts available readily like "DataNode Heap Usage", "NameNode Heap Usage(Weekly)", "NameNode Heap Usage (Daily)".

.

Re: Hello. How can I add customized Alert in Ambari for top 5 resource consuming jobs. The resources could be YARN Memory, Heap Memory or even Resource Manager Heap.

New Contributor
@Jay SenSharma

Thanks for the suggestion. Can that Python code be tweaked for knowing which Applications are using say more than 30% of the Queue? And the script should be in the Admin's login on the Edge node?

Thanks again.

Re: Hello. How can I add customized Alert in Ambari for top 5 resource consuming jobs. The resources could be YARN Memory, Heap Memory or even Resource Manager Heap.

Super Mentor

@Abhishek Vadnerkar

Python allows us to run the native OS commands as well as third party commands (like yarn, hdfs ..etc) So if you can extract the details of Yarn/Hadoop using the native commands then you can simply invoke those commands using python as well.

Please see an example: http://www.learn4master.com/big-data/hadoop/run-hadoop-command-in-python

You can also write some simple shell scripts to use the yarn / hadoop commands in it and then you can simply invoke that shell script from Python as well : http://www.learn4master.com/programming-language/shell/run-external-shell-command-in-python

.

.