Unable to start/restart standby namenode smoothly using Ambari

[#1]

The reason why Ambari is unable to start Namenode smoothly is bug and below is the workaround. Issue got fixed permanently in Ambari 2.5.x.

Few lines of Error message from Ambari Ops logs:

File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/decorator.py",
line 55, in wrapper  return function(*args,
**kwargs)  File
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py",
line 562, in is_this_namenode_active  raise Fail(format("The
NameNode {namenode_id} is not listed as Active or Standby, waiting..."))resource_management.core.exceptions.Fail: The
NameNode nn2 is not listed as Active or Standby, waiting...

ROOT CAUSE:

https://issues.apache.org/jira/browse/AMBARI-18786
RESOLUTION:

Increase the timeout in /var/lib/ambari-server/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py

from this;

@retry(times=5, sleep_time=5, backoff_factor=2,err_class=Fail)

to this;

@retry(times=25, sleep_time=25, backoff_factor=2,err_class=Fail)