Support Questions
Find answers, ask questions, and share your expertise

Name Node services fail over

Expert Contributor

I need to know if there was any failover happened on name node services. Where can I get this information.

Is there any REST API ?

1 ACCEPTED SOLUTION

Accepted Solutions

Expert Contributor

Fount out that this is available in zkfc logs which will be present in the name node.

For some reason these logs are not present in my name node boxes, so probably I need to restart the ZKFC services.

Thanks

Kumar

View solution in original post

3 REPLIES 3

Super Mentor

@Kumar Veerappan


The Following kind of Ambari API call can show which NameNode is Active and which one is StandBy.


Syntax:

curl -iv -u admin:admin -H "X-Requested-By: ambari" -X GET  "http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/components/NAMENODE?ServiceComponentInfo/category=MASTER&fields=ServiceComponentInfo/service_name,host_components/metrics/dfs/FSNamesystem/HAState"


Example:

curl -iv -u admin:admin -H "X-Requested-By: ambari" -X GET  "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/components/NAMENODE?ServiceComponentInfo/category=MASTER&fields=ServiceComponentInfo/service_name,host_components/metrics/dfs/FSNamesystem/HAState"

.

Example Output of the above call might look like:

{
  "href" : "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/components/NAMENODE?ServiceComponentInfo/category=MASTER&fields=ServiceComponentInfo/service_name,host_components/metrics/dfs/FSNamesystem/HAState",
  "ServiceComponentInfo" : {
    "category" : "MASTER",
    "cluster_name" : "plain_ambari",
    "component_name" : "NAMENODE",
    "service_name" : "HDFS"
  },
  "host_components" : [
    {
      "href" : "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/hosts/amb25101.example.com/host_components/NAMENODE",
      "HostRoles" : {
        "cluster_name" : "plain_ambari",
        "component_name" : "NAMENODE",
        "host_name" : "amb25101.example.com"
      },
      "metrics" : {
        "dfs" : {
          "FSNamesystem" : {
            "HAState" : "active"
          }
        }
      }
    },
    {
      "href" : "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/hosts/amb25102.example.com/host_components/NAMENODE",
      "HostRoles" : {
        "cluster_name" : "plain_ambari",
        "component_name" : "NAMENODE",
        "host_name" : "amb25102.example.com"
      },
      "metrics" : {
        "dfs" : {
          "FSNamesystem" : {
            "HAState" : "standby"
          }
        }
      }
    }
  ]
* Connection #0 to host amb25101.example.com left intact
}

.

Expert Contributor

@Kumar Veerappan

You will have to create an watcher/alert scripts that identifies which NN is active and alert/email if NN flips.

Namenode service provides JMX which has information on which NameNode is active. Your watcher script can query this data to identify if NN failovers.

name" : "Hadoop:service=NameNode,name=NameNodeStatus",
    "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
    "State" : "active",
    "NNRole" : "NameNode",
    "HostAndPort" : "host1:8020

Expert Contributor

Fount out that this is available in zkfc logs which will be present in the name node.

For some reason these logs are not present in my name node boxes, so probably I need to restart the ZKFC services.

Thanks

Kumar

View solution in original post