<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>question hbase Master failed to become active master in Support Questions</title>
    <link>https://community.cloudera.com/t5/Support-Questions/hbase-Master-failed-to-become-active-master/m-p/58933#M55098</link>
    <description>&lt;P&gt;My primary master died due to scheduled maintenance and my backup master failed to kick in&amp;nbsp;&lt;/P&gt;&lt;P&gt;CM Agent has tried to start it up but could not initialize the namespace table&amp;nbsp;&lt;/P&gt;&lt;P&gt;After several manual efforts where I followed&amp;nbsp;&lt;A href="https://community.cloudera.com/t5/Storage-Random-Access-HDFS/HBase-Master-Failed-to-become-active-master/m-p/27186#M1225" target="_blank"&gt;https://community.cloudera.com/t5/Storage-Random-Access-HDFS/HBase-Master-Failed-to-become-active-master/m-p/27186#M1225&lt;/A&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;I did the following only&amp;nbsp;&lt;/P&gt;&lt;PRE&gt;rmr /hbase/meta-region-server
rmr /hbase/rs
rmr /hbase/splitWAL
rmr /hbase/backup-masters
rmr /hbase/table-lock
rmr /hbase/flush-table-proc
rmr /hbase/region-in-transition
rmr /hbase/running
rmr /hbase/balancer 
rmr /hbase/recovering-regions
 rmr /hbase/draining
 rmr /hbase/namespace 
 rmr /hbase/hbaseid
 rmr /hbase/table&lt;/PRE&gt;&lt;P&gt;I got the a master to come up after setting&amp;nbsp;hbase.master.namespace.init.timeout to some absurd value&lt;/P&gt;&lt;P&gt;I see the master registering dead region servers (though I cannot find where it pick them up, not in the WAL, Archive or data)&lt;/P&gt;&lt;P&gt;and I see the master registering the following&amp;nbsp;&lt;/P&gt;&lt;P&gt;Starting namespace manager (since 1hrs, 20mins, 5sec ago)&lt;/P&gt;&lt;P&gt;even though cloudera manager shows healthy&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;list the catalog in hbase shell gives me the following error&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;PRE&gt;hbase(main):004:0&amp;gt; list
TABLE

ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
    at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2373)
    at org.apache.hadoop.hbase.master.MasterRpcServices.getTableNames(MasterRpcServices.java:907)
    at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55650)
    at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2182)
    at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
    at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
    at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)&lt;/PRE&gt;&lt;P&gt;fsck /hbase -files -blocks shows healthy&lt;/P&gt;&lt;P&gt;hbck &amp;nbsp;shows zero inconsistencies&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;I am on version&amp;nbsp;&lt;/P&gt;&lt;P&gt;Hadoop 2.6.0+cdh5.11.1+2400&lt;BR /&gt;HBase 1.2.0+cdh5.11.1+319&lt;/P&gt;&lt;P&gt;I did have a master colocated with a region server and was wondering if I ran into this&lt;/P&gt;&lt;P&gt;&lt;A href="https://issues.apache.org/jira/browse/HBASE-14861" target="_blank"&gt;https://issues.apache.org/jira/browse/HBASE-14861&lt;/A&gt;&lt;/P&gt;&lt;P&gt;and then this&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;A href="https://issues.apache.org/jira/browse/HBASE-14664" target="_blank"&gt;https://issues.apache.org/jira/browse/HBASE-14664&lt;/A&gt;&lt;/P&gt;&lt;P&gt;as the cause of the failed backup kicking in&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;But I cannot determine why I am getting&lt;SPAN&gt;&amp;nbsp;namespace manager would take so long to initialize&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
    <pubDate>Fri, 16 Sep 2022 12:06:12 GMT</pubDate>
    <dc:creator>scratch28</dc:creator>
    <dc:date>2022-09-16T12:06:12Z</dc:date>
  </channel>
</rss>

