Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

how to identify alert from kafka service in ambari by REST API

avatar

hi all

 

we want to identify kafka alert from ambari by rest api

 

Example when we get alert

 

Capture.PNG

 

what is the AMBARI REST API in order to verify alert on kafka broker01 or kafka broker02 , etc 

Michael-Bronson
1 ACCEPTED SOLUTION

avatar
Master Mentor

@mike_bronson7 

Yes, you are right if the alert state is "OK" means the service is running well usually. If it is WARNING/CRITICAL then we need to look at the alert text and alert host to find out why and in which host the alert is in that state.


Basically the Kafka "host" where the alert was triggered, The "state" of the alert like CRITICAL,OK,WARNING and then Alert "text" are usually the important parts of an alert which gives us a good idea on what is happening.

So you can capture those selected output using:

# curl -u admin:admin -H "X-Requested-By: ambari" -X GET "http://$AMBARI_HOST:8080/api/v1/clusters/NewCluster/alerts?fields=Alert/host_name,Alert/host_name,Alert/state,Alert/text&Alert/service_name=KAFKA"

.

Example Output:

# curl -u admin:admin -H "X-Requested-By: ambari" -X GET "<a href="http://newhwx1.example.com:8080/api/v1/clusters/$CLUSTER_NAME/alerts?fields=Alert/host_name,Alert/host_name,Alert/state,Alert/text&Alert/service_name=KAFKA" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/$CLUSTER_NAME/alerts?fields=Alert/host_name,Alert/host_name,Alert/state,Alert/text&Alert/service_name=KAFKA</a>" 
{
  "href" : "<a href="http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts?fields=Alert/host_name,Alert/host_name,Alert/state,Alert/text&Alert/service_name=KAFKA" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts?fields=Alert/host_name,Alert/host_name,Alert/state,Alert/text&Alert/service_name=KAFKA</a>",
  "items" : [
    {
      "href" : "<a href="http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/704" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/704</a>",
      "Alert" : {
        "cluster_name" : "NewCluster",
        "definition_id" : 401,
        "definition_name" : "kafka_broker_process",
        "host_name" : "newhwx3.example.com",
        "id" : 704,
        "service_name" : "KAFKA",
        "state" : "OK",
        "text" : "TCP OK - 0.000s response on port 6667"
      }
    },
    {
      "href" : "<a href="http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/1201" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/1201</a>",
      "Alert" : {
        "cluster_name" : "NewCluster",
        "definition_id" : 401,
        "definition_name" : "kafka_broker_process",
        "host_name" : "newhwx5.example.com",
        "id" : 1201,
        "service_name" : "KAFKA",
        "state" : "CRITICAL",
        "text" : "Connection failed: [Errno 111] Connection refused to newhwx5.example.com:6667"
      }
    }
  ]
}

.

 

View solution in original post

3 REPLIES 3

avatar
Master Mentor

@mike_bronson7 

You can try something like this:

 

# curl -u admin:admin -H "X-Requested-By: ambari" -X GET "http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/alerts?fields=*&Alert/service_name=KAFKA"

 

.

Example Output:

 

# curl -u admin:admin -H "X-Requested-By: ambari" -X GET "<a href="http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts?fields=*&Alert/service_name=KAFKA" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts?fields=*&Alert/service_name=KAFKA</a>"
{
  "href" : "<a href="http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts?fields=*&Alert/service_name=KAFKA" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts?fields=*&Alert/service_name=KAFKA</a>",
  "items" : [
    {
      "href" : "<a href="http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/704" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/704</a>",
      "Alert" : {
        "cluster_name" : "NewCluster",
        "component_name" : "KAFKA_BROKER",
        "definition_id" : 401,
        "definition_name" : "kafka_broker_process",
        "firmness" : "HARD",
        "host_name" : "newhwx3.example.com",
        "id" : 704,
        "instance" : null,
        "label" : "Kafka Broker Process",
        "latest_timestamp" : 1575403190535,
        "maintenance_state" : "OFF",
        "occurrences" : 14,
        "original_timestamp" : 1575402410385,
        "repeat_tolerance" : 1,
        "repeat_tolerance_remaining" : 0,
        "scope" : "HOST",
        "service_name" : "KAFKA",
        "state" : "OK",
        "text" : "TCP OK - 0.000s response on port 6667"
      }
    },
    {
      "href" : "<a href="http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/1201" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/1201</a>",
      "Alert" : {
        "cluster_name" : "NewCluster",
        "component_name" : "KAFKA_BROKER",
        "definition_id" : 401,
        "definition_name" : "kafka_broker_process",
        "firmness" : "HARD",
        "host_name" : "newhwx5.example.com",
        "id" : 1201,
        "instance" : null,
        "label" : "Kafka Broker Process",
        "latest_timestamp" : 1575403167289,
        "maintenance_state" : "OFF",
        "occurrences" : 12,
        "original_timestamp" : 1575402507311,
        "repeat_tolerance" : 1,
        "repeat_tolerance_remaining" : 0,
        "scope" : "HOST",
        "service_name" : "KAFKA",
        "state" : "CRITICAL",
        "text" : "Connection failed: [Errno 111] Connection refused to newhwx5.example.com:6667"
      }
    }
  ]
}

 

 

 

 

.

 

avatar

Dear Jay

 

so from your example , I understand that we need to capture the - "state" from the output for each kafka broker , while expected output should be OK

 

am I right until now?

Michael-Bronson

avatar
Master Mentor

@mike_bronson7 

Yes, you are right if the alert state is "OK" means the service is running well usually. If it is WARNING/CRITICAL then we need to look at the alert text and alert host to find out why and in which host the alert is in that state.


Basically the Kafka "host" where the alert was triggered, The "state" of the alert like CRITICAL,OK,WARNING and then Alert "text" are usually the important parts of an alert which gives us a good idea on what is happening.

So you can capture those selected output using:

# curl -u admin:admin -H "X-Requested-By: ambari" -X GET "http://$AMBARI_HOST:8080/api/v1/clusters/NewCluster/alerts?fields=Alert/host_name,Alert/host_name,Alert/state,Alert/text&Alert/service_name=KAFKA"

.

Example Output:

# curl -u admin:admin -H "X-Requested-By: ambari" -X GET "<a href="http://newhwx1.example.com:8080/api/v1/clusters/$CLUSTER_NAME/alerts?fields=Alert/host_name,Alert/host_name,Alert/state,Alert/text&Alert/service_name=KAFKA" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/$CLUSTER_NAME/alerts?fields=Alert/host_name,Alert/host_name,Alert/state,Alert/text&Alert/service_name=KAFKA</a>" 
{
  "href" : "<a href="http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts?fields=Alert/host_name,Alert/host_name,Alert/state,Alert/text&Alert/service_name=KAFKA" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts?fields=Alert/host_name,Alert/host_name,Alert/state,Alert/text&Alert/service_name=KAFKA</a>",
  "items" : [
    {
      "href" : "<a href="http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/704" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/704</a>",
      "Alert" : {
        "cluster_name" : "NewCluster",
        "definition_id" : 401,
        "definition_name" : "kafka_broker_process",
        "host_name" : "newhwx3.example.com",
        "id" : 704,
        "service_name" : "KAFKA",
        "state" : "OK",
        "text" : "TCP OK - 0.000s response on port 6667"
      }
    },
    {
      "href" : "<a href="http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/1201" target="_blank">http://newhwx1.example.com:8080/api/v1/clusters/NewCluster/alerts/1201</a>",
      "Alert" : {
        "cluster_name" : "NewCluster",
        "definition_id" : 401,
        "definition_name" : "kafka_broker_process",
        "host_name" : "newhwx5.example.com",
        "id" : 1201,
        "service_name" : "KAFKA",
        "state" : "CRITICAL",
        "text" : "Connection failed: [Errno 111] Connection refused to newhwx5.example.com:6667"
      }
    }
  ]
}

.