Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NameNode failing unable to load it's state Active/Standby

NameNode failing unable to load it's state Active/Standby

New Contributor

Please find the below logs while namenode reboot.

sterr:

  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/jmx.py", line 42, in get_value_from_jmx
    return data_dict["beans"][0][property]
IndexError: list index out of range
2018-11-05 13:27:52,753 - Getting jmx metrics from NN failed. URL: http://sjdcdlake02.np1.ril.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/jmx.py", line 42, in get_value_from_jmx
    return data_dict["beans"][0][property]
IndexError: list index out of range

 Python script has been killed due to timeout after waiting 1800 secs

stdout:

2018-11-05 13:27:55,682 - call returned (255, '18/11/05 13:27:55 INFO ipc.Client: Retrying connect to server: sjdcdlake02.np1.ril.com/10.21.51.76:8020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=1, sleepTime=1000 MILLISECONDS)\nOperation failed: Call From sjdcdlake02.np1.ril.com/10.21.51.76 to sjdcdlake02.np1.ril.com:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused')
2018-11-05 13:27:55,683 - NameNode HA states: active_namenodes = [(u'nn1', 'sjdcdlake01.np1.ril.com:50070')], standby_namenodes = [], unknown_namenodes = [(u'nn2', 'sjdcdlake02.np1.ril.com:50070')]
2018-11-05 13:27:55,684 - Will retry 8 time(s), caught exception: The NameNode nn2 is not listed as Active or Standby, waiting.... Sleeping for 25 sec(s)