As of HDFS CM5.12.1, the standby namenode is busy under high availability

[#1]

As of HDFS CM5.12.1, the standby namenode is busy under high availability,

I'm not sure if the busy standby namenode is hot standby, so please help me see how to fix it. The busy state of the standby namenode restores it to the start state.

The action buttons of the standby namenode have all turned gray, making it impossible to restart, stop, and start the namenode.

[#2]

@ns2, Welcome to our community! To help you get the best possible answer, I have tagged our HDFS experts @willx @ChethanYM @SVB who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

[#3]

Some steps to help narrow down the issue, if possible please attach outputs or the answer to the below items:

1. In order to know if the issue is from CM or HDFS, can you please check if the Standby NN's process is existing or not by running:

ps -ef|grep -i namenode

2. Please check do we have any ERROR/WARN in the latest Standby NN, Are there any GC pause issues detected in the Standby NN's log? Attaching the errors may help us know the issue better.

3. Please check the status of cloudera-scm-agent by running the below commands, make sure the agent is Active:

systemctl status cloudera-scm-agent

4. How about the other services on this host, they are all good but only SNN has this issue?

5. Please try to open NN webUI and SNN webUI from the browser, if SNN is up and running, the webUI should be good:

The default webUI port is 9870

http://NN_ip:9870/dfshealth.html#tab-overview

6. Please check if cpu utilization and memory utilization are sufficient in this SNN host.

7. When and how did this issue happen, did it happen after the restarting?

[#4 · Accepted solution]

First of all, thank you. All the above 7 points have been considered, and now the problem has been solved: modify the status of this snn in the scm database.

[#5]

@ns2, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  

[#6]

@ns2,  I'm happy to see you resolved your issue. Could you please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future?