I'm running on HDP 2.3 hosted on AWS and Ambari version is 2.1
NameNode HA is enabled and I can see two name nodes as active in the Ambari. Attached screenshot for reference.
ZookeeperFailControllers are running fine.
When I Start HDFS from command line using the below command, the namenode service on the node from which I execute the command is working and causing the other name node to stop.
sudo -u hdfs /usr/hdp/22.214.171.124-3796/hadoop/sbin/hadoop-daemon.sh start namenode
I formatted Zookeeper and restartwed HDFS, still no luck.
When I open zkCli and checked, there is no ActiveStandbyElectorLock and I found the ephermal node value as 0*0
Any help would be appreciated.
Please refer to api calls in the following thread to check ha_state of the name node and make sure it is correctly set. If not, please attach the response and we can look into the problem further.
The best solution is to stop all the services and start Zookeeper only. And then go to CLI where your namenode is present. Run the command => hdfs zkfc -formatZK (you'll be asked for an option as Y or N, input Y) and through ambari start all the services again. Your Namenode will be in Active-Standby mode again!