Support Questions
Find answers, ask questions, and share your expertise

Active Name Nodes behavior difference in HA Cluster

iam using hadoop apache 2.7.1 with zookeeper quorum in order to achieve automatic failover on centos 7

and every thing is done ok but i am facing this strange problem

supposing my active two name nodes are mn1 and mn2

when mn1 is off so because of automatic failover mn2 is active

and iam using webhdfs requests which are directed to mn2 by client and the performance is greate

but in the reverse status i mean when mn2 is off and mn1 becomes active and webhdfs requests are directed to mn1 the performance got stack until mn2 is on again even though if we turn on a mock mn2 i mean a host with same name with no services

notice1: mn1 and mn2 has same configurations and all cluster nodes are vms which are connected to virtual switch so there are no network issues to worry about

notice2:there are no difference between two active name nodes if we issue hdfs commands the difference only in webhdfs interface requests (in curl requests)

what could be possible reasons of this difference in performance between two active name nodes ?


Re: Active Name Nodes behavior difference in HA Cluster alshiekh

possibly your retry policy is set too high, could you please check what the value for dfs.client.retry.policy.spec is?

Re: Active Name Nodes behavior difference in HA Cluster

it doesn't succeed any other options?

Re: Active Name Nodes behavior difference in HA Cluster

in order to analyze my problem and because i am working on virtual machines i copied second name node virtual machine (i mean the name node which behaves well and change it's ip and host name to be the same of first name node which is slower so i have same network and software configuration) and i got the same results, what a strange state it's related to hostname only , i will mention my problem again when mn1 is off and mn2 is active and on and ever thing is ok ,but when mn2 is off and mn1 is active , mn1 responds slower as data nodes send requests to this dead mn2 name node host name and i don't know why and that's of ocourse when issuing webhdfs requests only and not on hdfs commands eventhough my requests are directed to active name node from client application