question Re: Name Node services fail over in Support Questions

Name Node services fail over

kums — Tue, 26 Sep 2017 23:07:40 GMT

I need to know if there was any failover happened on name node services. Where can I get this information.

Is there any REST API ?

Re: Name Node services fail over

jsensharma — Tue, 26 Sep 2017 23:43:00 GMT

@Kumar Veerappan

The Following kind of Ambari API call can show which NameNode is Active and which one is StandBy.

Syntax:

curl -iv -u admin:admin -H "X-Requested-By: ambari" -X GET  "http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/components/NAMENODE?ServiceComponentInfo/category=MASTER&fields=ServiceComponentInfo/service_name,host_components/metrics/dfs/FSNamesystem/HAState"

Example:

curl -iv -u admin:admin -H "X-Requested-By: ambari" -X GET  "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/components/NAMENODE?ServiceComponentInfo/category=MASTER&fields=ServiceComponentInfo/service_name,host_components/metrics/dfs/FSNamesystem/HAState"

Example Output of the above call might look like:

{
  "href" : "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/components/NAMENODE?ServiceComponentInfo/category=MASTER&fields=ServiceComponentInfo/service_name,host_components/metrics/dfs/FSNamesystem/HAState",
  "ServiceComponentInfo" : {
    "category" : "MASTER",
    "cluster_name" : "plain_ambari",
    "component_name" : "NAMENODE",
    "service_name" : "HDFS"
  },
  "host_components" : [
    {
      "href" : "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/hosts/amb25101.example.com/host_components/NAMENODE",
      "HostRoles" : {
        "cluster_name" : "plain_ambari",
        "component_name" : "NAMENODE",
        "host_name" : "amb25101.example.com"
      },
      "metrics" : {
        "dfs" : {
          "FSNamesystem" : {
            "HAState" : "active"
          }
        }
      }
    },
    {
      "href" : "http://amb25101.example.com:8080/api/v1/clusters/plain_ambari/hosts/amb25102.example.com/host_components/NAMENODE",
      "HostRoles" : {
        "cluster_name" : "plain_ambari",
        "component_name" : "NAMENODE",
        "host_name" : "amb25102.example.com"
      },
      "metrics" : {
        "dfs" : {
          "FSNamesystem" : {
            "HAState" : "standby"
          }
        }
      }
    }
  ]
* Connection #0 to host amb25101.example.com left intact
}

Re: Name Node services fail over

PranayV — Wed, 27 Sep 2017 08:12:07 GMT

@Kumar Veerappan

You will have to create an watcher/alert scripts that identifies which NN is active and alert/email if NN flips.

Namenode service provides JMX which has information on which NameNode is active. Your watcher script can query this data to identify if NN failovers.

name" : "Hadoop:service=NameNode,name=NameNodeStatus",
    "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
    "State" : "active",
    "NNRole" : "NameNode",
    "HostAndPort" : "host1:8020

Re: Name Node services fail over

kums — Thu, 28 Sep 2017 20:55:25 GMT

Fount out that this is available in zkfc logs which will be present in the name node.

For some reason these logs are not present in my name node boxes, so probably I need to restart the ZKFC services.

Thanks

Kumar