Bosun is a time series alerting framework developed by Stack Exchange. Scollector is a metric collection agent. Learn more at bosun.org.
Grafana 3.0+ has a bosun plugin and Ambari should be able to support this. Care to open a Apache Jira for this as desired feature?