Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDFS NameNode won't leave safemode

avatar
Explorer

Hello,

- I've setup a Sandbox HDP 3.0.1.
- Determine Network Adapter of Your VirtualBox Sandbox (Bridged Adapter, IP:192.168.1.37).
- I edited the hosts file (on my computer, windows) and assigned the domain sandbox-hdp.hortonworks.com to the IP address of 192.168.1.37, but When I start all services via the Ambari UI, I receive the following error message:

 

safemode: Call From sandbox-hdp.hortonworks.com/172.18.0.2 to sandbox-hdp.hortonworks.com:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused


2019-09-29 18:25:10,968 - Retrying after 10 seconds. Reason: Execution of '/usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs dfsadmin -fs hdfs://sandbox-hdp.hortonworks.com:8020 -safemode get | grep 'Safe mode is OFF'' returned 1. safemode: Call From sandbox-hdp.hortonworks.com/172.18.0.2 to sandbox-hdp.hortonworks.com:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused.

 

I do not know how to fix the error. Who knows, please just help, thank you!

1 ACCEPTED SOLUTION

avatar
New Contributor

At the startup of NameNode it loads the file system namespace from the last saved fsimage into its main memory and the edits log file showbox apk. Then, merges edits log file on fsimage and results in new file system namespace.

View solution in original post

14 REPLIES 14

avatar
Master Mentor

@ThanhP 

I can see you are accessing the web shell UI on 192.168.1.37 yet the sandbox is pointing to 172.18.0.2:8020. Now if you type in your browser http://192.168.1.37:1080/splash2.html you should land on the splash screen
Choose the Ambari UI link , this should give you the normal Ambari user UI if you reset your password before as I had stated please use that admin/[password] combination and you will realize that none of the components are started HDFS, YARN, MR, HIVE, HBASE etc Note: some components are in maintenance mode unless you want any of them beware of the dependencies e.g ATLAS must have HBASE running etc.. that could explain why you got the error "Failed on connection error java.net.Connection"

At a certain point during your startup, "Timeline Service V1.5" will error out with the message to do with "safemode on" get to the shell as hdfs

$ hdfs dfsadmin -safemode get
This should indicate it's ON so get it out of safemode

$ hdfs dfsadmin -safemode leave

And again use the option Ambari start all it will pick up from the last point of failure, after the components startup you can simply access them eg the name node UI will automatically point to http://sandbox-hdp.hortonworks.com:50070/dfshealth.html#tab-overview

Please do that an revert

avatar
Explorer

Hi @Shelton 

Before, reading your instructions, I tried running this command

sudo -u hdfs hdfs namenode -format

After executing the above command, NAMENODE has run

Thank you for your enthusiastic help111.JPG

avatar
Master Mentor

@ThanhP 

Good everything is perfect for you now 🙂

You ONLY execute sudo -u hdfs hdfs namenode -format  as a last resort because it's dangerous and not recommended to run that on production cluster as that [re-initializes] formats your Namenode hence deleting all your metadata stored on the NameNode.

 

Having said that the answer you accepted can't help a member who encounters the same issue "HDFS NameNode won't leave safemode" maybe you should un-accept it and accept your own answer as it's the more realistic answer .
Happy hadooping 

 

avatar
Explorer

@Shelton 

Thanks to your reminder, I will learn from experience so as not to affect the community. Thanks for your help!

avatar
New Contributor

@Shelton 

I could not start namenode with ambari ui when setting safemode to off mode.

Screenshot 2023-10-01 190748.png

Please could you helpme out!

And also while setting safemode to 'ON'  does not allow me to do task.