https://community.cloudera.com/t5/Support-Questions/HDFS-NameNode-won-t-leave-safemode/m-p/195773#M157829 <P>I've setup a HDP 2.6.3 Hadoop Cluster with Ambari 2.5.2 (was HDP 2.5.2 and Ambari 2.4.2 earlier, but had the same situation). When I <STRONG>start all services</STRONG> via the Ambari UI, the process stucks at starting the NameNode service. The output always sais:</P><PRE>2017-11-15 09:23:25,594 - Waiting for this NameNode to leave Safemode due to the following conditions: HA: False, isActive: True, upgradeType: None 2017-11-15 09:23:25,594 - Waiting up to 19 minutes for the NameNode to leave Safemode... 2017-11-15 09:23:25,595 - Execute['/usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs dfsadmin -fs hdfs://my-host.com:8020 -safemode get | grep 'Safe mode is OFF''] {'logoutput': True, 'tries': 115, 'user': 'hdfs', 'try_sleep': 10} 2017-11-15 09:23:27,811 - Retrying after 10 seconds. Reason: Execution of '/usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs dfsadmin -fs hdfs://my-host.com:8020 -safemode get | grep 'Safe mode is OFF'' returned 1. 2017-11-15 09:23:40,148 - Retrying after 10 seconds. Reason: Execution of '/usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs dfsadmin -fs hdfs://my-host.com:8020 -safemode get | grep 'Safe mode is OFF'' returned 1. 2017-11-15 09:23:52,525 - Retrying after 10 seconds. Reason: Execution of '/usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs dfsadmin -fs hdfs://my-host.com:8020 -safemode get | grep 'Safe mode is OFF'' returned 1. 2017-11-15 09:24:04,853 - Retrying after 10 seconds. Reason: Execution of '/usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs dfsadmin -fs hdfs://my-host.com:8020 -safemode get | grep 'Safe mode is OFF'' returned 1. 2017-11-15 09:24:17,238 - Retrying after 10 seconds. Reason: Execution of '/usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs dfsadmin -fs hdfs://my-host.com:8020 -safemode get | grep 'Safe mode is OFF'' returned 1. 2017-11-15 09:24:29,566 - Retrying after 10 seconds. Reason: Execution of '/usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs dfsadmin -fs hdfs://my-host.com:8020 -safemode get | grep 'Safe mode is OFF'' returned 1. ...</PRE><P>I think this issue occurs since I setup the cluster ~6 months ago.</P><P>I always try to leave the safemode manually using the command:</P><PRE>hdfs dfsadmin -safemode leave</PRE><P>But this command doesn't help very often, mostly it shows that safemode is still ON. So I have to force the safemode exit with</P><PRE>hdfs dfsadmin -safemode forceExit</PRE><P>Afterwards the NameNode start resumes and all other services start also fine. When I forget to type the forceExit command, the NameNode service start times out and the following service starts fail also.</P><P>Can someone explain, why Ambari / NameNode can't leave the safemode automatically? What could here be the reason? </P><P>Here a screenshot of my HDFS overview on Ambari, after a successful start of the services (after forceExit):</P><P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="43602-unbenannt.png" style="width: 1192px;"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/17984iF63768CA35C5BCE0/image-size/medium?v=v2&amp;px=400" role="button" title="43602-unbenannt.png" alt="43602-unbenannt.png" /></span></P><P>Any help would be appreciated, thank you!</P> Sun, 18 Aug 2019 06:34:42 GMT dmueller1607 2019-08-18T06:34:42Z