Support Questions

Find answers, ask questions, and share your expertise

Who agreed with this solution

avatar
Super Collaborator

Hello @loridigia ,

 

It seems due to the outage there would be multiple ServerCrashProcedures created for the Regionservers. The Dead region severs with same names are different instances of the Region servers with a different epoch timestamp. As the Hbase Master was also down, it might be possible that it was not able to process the expiration of the Region servers. You might see some Crash procedures waiting to be finished under "Procedures & Locks" section of the Active Hbase Master Web UI.

As you have already solved this issue in the past involving zookeeper. I guess you can try this :

1. Stop Hbase

2. Login to zookeeper using #hbase zkcli ( with a valid hbase ticket )

3. Delete the /hbase-secure znode. rmr /hbase-secure

4. Sideline the entries under HDFS dir. hdfs dfs -mv /hbase/MasterProcWALs/*  /tmp. ( Not sure if this was done earlier )

5. Start Hbase

View solution in original post

Who agreed with this solution