Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Name Node services fail over

avatar
Expert Contributor

I need to know if there was any failover happened on name node services. Where can I get this information.

Is there any REST API ?

1 ACCEPTED SOLUTION

avatar
Expert Contributor

Fount out that this is available in zkfc logs which will be present in the name node.

For some reason these logs are not present in my name node boxes, so probably I need to restart the ZKFC services.

Thanks

Kumar

View solution in original post

3 REPLIES 3

avatar
Master Mentor

@Kumar Veerappan


The Following kind of Ambari API call can show which NameNode is Active and which one is StandBy.


Syntax:

curl -iv -u admin:admin -H "X-Requested-By: ambari" -X GET  "http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/components/NAMENODE?ServiceComponentInfo/category=MASTER&fields=ServiceComponentInfo/service_name,host_components/metrics/dfs/FSNamesystem/HAState"


Example:

curl -iv -u admin:admin -H "X-Requested-By: ambari" -X GET  "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/components/NAMENODE?ServiceComponentInfo/category=MASTER&fields=ServiceComponentInfo/service_name,host_components/metrics/dfs/FSNamesystem/HAState"

.

Example Output of the above call might look like:

{
  "href" : "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/components/NAMENODE?ServiceComponentInfo/category=MASTER&fields=ServiceComponentInfo/service_name,host_components/metrics/dfs/FSNamesystem/HAState",
  "ServiceComponentInfo" : {
    "category" : "MASTER",
    "cluster_name" : "plain_ambari",
    "component_name" : "NAMENODE",
    "service_name" : "HDFS"
  },
  "host_components" : [
    {
      "href" : "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/hosts/amb25101.example.com/host_components/NAMENODE",
      "HostRoles" : {
        "cluster_name" : "plain_ambari",
        "component_name" : "NAMENODE",
        "host_name" : "amb25101.example.com"
      },
      "metrics" : {
        "dfs" : {
          "FSNamesystem" : {
            "HAState" : "active"
          }
        }
      }
    },
    {
      "href" : "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/hosts/amb25102.example.com/host_components/NAMENODE",
      "HostRoles" : {
        "cluster_name" : "plain_ambari",
        "component_name" : "NAMENODE",
        "host_name" : "amb25102.example.com"
      },
      "metrics" : {
        "dfs" : {
          "FSNamesystem" : {
            "HAState" : "standby"
          }
        }
      }
    }
  ]
* Connection #0 to host amb25101.example.com left intact
}

.

avatar
Super Collaborator

@Kumar Veerappan

You will have to create an watcher/alert scripts that identifies which NN is active and alert/email if NN flips.

Namenode service provides JMX which has information on which NameNode is active. Your watcher script can query this data to identify if NN failovers.

name" : "Hadoop:service=NameNode,name=NameNodeStatus",
    "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
    "State" : "active",
    "NNRole" : "NameNode",
    "HostAndPort" : "host1:8020

avatar
Expert Contributor

Fount out that this is available in zkfc logs which will be present in the name node.

For some reason these logs are not present in my name node boxes, so probably I need to restart the ZKFC services.

Thanks

Kumar