To help you, we'd need some more information: - Which version of HDP are you actually running on currently? Is it 2.5.3.0-37? - Can you post the entire output from the install command? - What is the content of /usr/hdp on this host which is having trouble? ... View more

Although this will technically work, there is a supported way of doing this. The Falcon alert definition can specify the parameter to monitor for determining whether to use HTTP or HTTPS: { "name": "falcon_server_webui", "label": "Falcon Server Web UI", "description": "This host-level alert is triggered if the Falcon Server Web UI is unreachable.", "interval": 1, "scope": "ANY", "enabled": true, "source": { "type": "WEB", "uri": { "http": "{{falcon-env/falcon_port}}", "https": "{{falcon-env/falcon_port}}", "https_property": "{{hdfs-site/falcon.enableTLS}}", "https_property_value": "true", "default_port": 15000, "kerberos_keytab": "{{falcon-startup.properties/*.falcon.http.authentication.kerberos.keytab}}", "kerberos_principal": "{{falcon-startup.properties/*.falcon.http.authentication.kerberos.principal}}", "connection_timeout": 5 }, "reporting": { "ok": { "text": "HTTP {0} response in {2:.3f}s" }, "warning": { "text": "HTTP {0} response from {1} in {2:.3f}s ({3})" }, "critical": { "text": "Connection failed to {1} ({3})" } } } } Falcon should respect the port, regardless of plaintext vs encryption. However, this way, the alert framework will understand whether to use plaintext or TLS. ... View more

Yes, my example is correct. There is no way to query directly for a specific property; you can only query by bean name. However, for alerts, we use a slash as a delimiter. The metric alert will remove the "VolumeFailuresTotal" and retrieve the "Hadoop:service=NameNode,name=FSNamesystemState" bean. Then it will extract the "VolumeFailuresTotal" metric. ... View more

Sure, you'd need to execute a POST to create the new alert: POST api/v1/clusters/<cluster-name>/alert_definitions { "AlertDefinition": { "component_name": "NAMENODE", "description": "This service-level alert is triggered if the total number of volume failures across the cluster is greater than the configured critical threshold.", "enabled": true, "help_url": null, "ignore_host": false, "interval": 2, "label": "NameNode Volume Failures", "name": "namenode_volume_failures", "scope": "ANY", "service_name": "HDFS", "source": { "jmx": { "property_list": [ "Hadoop:service=NameNode,name=FSNamesystemState/VolumeFailuresTotal" ], "value": "{0}" }, "reporting": { "ok": { "text": "There are {0} volume failures" }, "warning": { "text": "There are {0} volume failures", "value": 1 }, "critical": { "text": "There are {0} volume failures", "value": 1 }, "units": "Volume(s)" }, "type": "METRIC", "uri": { "http": "{{hdfs-site/dfs.namenode.http-address}}", "https": "{{hdfs-site/dfs.namenode.https-address}}", "https_property": "{{hdfs-site/dfs.http.policy}}", "https_property_value": "HTTPS_ONLY", "kerberos_keytab": "{{hdfs-site/dfs.web.authentication.kerberos.keytab}}", "kerberos_principal": "{{hdfs-site/dfs.web.authentication.kerberos.principal}}", "default_port": 0, "connection_timeout": 5, "high_availability": { "nameservice": "{{hdfs-site/dfs.internal.nameservices}}", "alias_key": "{{hdfs-site/dfs.ha.namenodes.{{ha-nameservice}}}}", "http_pattern": "{{hdfs-site/dfs.namenode.http-address.{{ha-nameservice}}.{{alias}}}}", "https_pattern": "{{hdfs-site/dfs.namenode.https-address.{{ha-nameservice}}.{{alias}}}}" } } } } } This will create a new METRIC alert which runs every 2 minutes. ... View more

Currently no, there is not. I believe there is a Jira open for changing how we send arguments to a script dispatcher so they are parameterized. Some fields, such as host name and component name, are not always present, so the current model simply omits them. ... View more

I think this goes back to the whole "dead is bad" theory. If I recall correctly, there was a metric Ambari was monitoring once on HBase - it was for "Dead RegionServers". We incorrectly assumed that "dead" was "bad". Because of this, while decommissioning a RegionServer, alerts would trigger (and not go away for a long time). In the end, it was determined that this metric wasn't really something which needed alerting on. HDFS is a little different - I believe that a DataNode is marked as stale if it hasn't reported in within 30 seconds and marked as dead if it hasn't reported within 1 minute. The problem here is that action is taken by the NameNode in this case - it will begin replicating blocks when it believes a DataNode is dead. So, we alert on it since it's something that is actively causing changes in the cluster data. The NameNode actually has metrics for differentiating "dead" vs "decommissiong dead": "NumLiveDataNodes": 3, "NumDeadDataNodes": 1, "NumDecomLiveDataNodes": 0, "NumDecomDeadDataNodes": 1, In the above example, Ambari won't worry about dead nodes which are marked as known decommissioning, but we will worry about this which are unexpected. ... View more