How to set alerts for yarn applications

[#1]

Hi ,

I would like to set up alerts ( mail alert) for the below events . How to achieve this using cloudera manager?

Whenever a job fails.

When a job is in Accepted state for more than 10 minutes.

[#2 · Accepted solution]

Hello @npdell,

Cloudera Manager trigger is what you need. You can create it at here:

CM --> YARN --> Status --> Create trigger --> Edit manually

Examples:

1) It will alert if there are more than 50 applications in pending state

Expression:

IF (select total_apps_pending_across_yarn_pools WHERE entityName=$SERVICENAME and LAST( total_apps_pending_across_yarn_pools) > 50) DO health:concerning

Metric Evaluation Window: 10 minutes

2) It will alert if more than 5 applications are failing

Expression:

IF (select total_apps_failed_rate_across_yarn_pools WHERE entityName=$SERVICENAME and LAST( total_apps_failed_rate_across_yarn_pools) > 5) DO health:concerning

Here is the documentation about CM triggers:

http://www.cloudera.com/documentation/enterprise/latest/topics/cm_dg_triggers.html

Gabor

[#3]

Maybe you can combine these two for a failed job alert:

• “Configuring Custom Alert Scripts”
• “Configuring Log Alerts”

My thought process is, that in the RM log, there is a very reliable pattern when a job fails (e.g. finalStatus=FAILED), so you could set up a Log Alert for that.