HDP cluster daily health report

[#1]

Hello,

I would like to check daily HDP cluster status and go through each components and send out the report to our group. What is the best way do it? Is there a command line script I can use?

[#2]

@Anpan K I think ambari API can be useful here. Try using the API command:

http://ambari:8080/api/v1/clusters/<cluster_name>?fields=Clusters/health_report

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

[#3]

@Anpan K More detailed information for per service/component level status on the following link:

https://community.hortonworks.com/questions/156898/%D7%99%D7%9D-to-capture-all-ambari-servicescompon...

[#4]

Thank you very much and it is really helpful. How can I find the status for each service? Any other commands like this?

[#5]

@Anpan K

You can make use fo Ambari API call to findout all the component status using a single API call. "INSTALLED" means not running and "STARTED" means running state for the components. For Client conponents like HDFS_CLIENT/HIVE_CLIENT ...etc we will always see "INSTALLED" state.

Example:

# curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://myambariserver.example.com:8080/api/v1/clusters/TestCluster/components?fields=ServiceComponen... | grep -A 2 component_name | awk '{print $NF}' > /tmp/requests_details.txt; sed -e '1,2d' -e s'/--//g'  -e 's/\n//g' -e 's/"//g' -e '/^$/d' /tmp/requests_details.txt | awk 'NR%3{printf "%s ",$0;next;}1' 

Example Output:

STARTED DATANODE, HDFS,
STARTED DATANODE, hdfcluster1.example.com,
STARTED DATANODE, hdfcluster2.example.com,
STARTED DATANODE, hdfcluster3.example.com,
STARTED DATANODE, hdfcluster4.example.com,
STARTED DRPC_SERVER, STORM,
STARTED HBASE_MASTER, hdfcluster4.example.com,
STARTED HBASE_REGIONSERVER, HBASE,
STARTED HBASE_REGIONSERVER, hdfcluster3.example.com,
STARTED HBASE_REGIONSERVER, hdfcluster4.example.com,
STARTED HCAT, HIVE,
STARTED HISTORYSERVER, hdfcluster2.example.com,
STARTED HIVE_CLIENT, HIVE,
STARTED HIVE_METASTORE, hdfcluster3.example.com,
STARTED HIVE_SERVER, HIVE,
STARTED HIVE_SERVER, hdfcluster3.example.com,
STARTED HIVE_SERVER_INTERACTIVE, HIVE,
STARTED HIVE_SERVER_INTERACTIVE, hdfcluster2.example.com,
STARTED HST_AGENT, hdfcluster1.example.com,
STARTED HST_SERVER, hdfcluster1.example.com,
STARTED INFRA_SOLR, hdfcluster4.example.com,
STARTED INFRA_SOLR_CLIENT, AMBARI_INFRA,
STARTED NAMENODE, HDFS,
STARTED NAMENODE, hdfcluster1.example.com,
STARTED NAMENODE, hdfcluster2.example.com,
STARTED RESOURCEMANAGER, hdfcluster2.example.com,
STARTED RESOURCEMANAGER, hdfcluster3.example.com,
STARTED SECONDARY_NAMENODE, HDFS,
.
.

.

In the above API call you will need to make the changes for the following values:

1. TestCluster : Replace this with your own cluster name.

2. admin:admin : Replace this with your own ambari admin username:password.

3. http://myambariserver.example.com:8080 Replace this URL with your own ambari server URL.

[#6]

Thank you and it is really helpful. The funny thing is in command line it shows as started, but in ambari gui it shows as down. I can see pid ID for /var/run/smartsense-activity-analyzer/activity-analyzer.pid

How does it check the status ? using /var/run/xxx/pid ?

[#7]

I can see the health report, but not sure which one is unhealthy. how do I check?

"health_report" : { "Host/stale_config" : 0, "Host/maintenance_state" : 13, "Host/host_state/HEALTHY" : 13, "Host/host_state/UNHEALTHY" : 0, "Host/host_state/HEARTBEAT_LOST" : 0, "Host/host_state/INIT" : 0, "Host/host_status/HEALTHY" : 12, "Host/host_status/UNHEALTHY" : 1, "Host/host_status/UNKNOWN" : 0, "Host/host_status/ALERT" : 0