Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cannot restart namenode: Failed to load an FSImage file

Cannot restart namenode: Failed to load an FSImage file

Explorer

I have Cluster Express with 1 master role, 1 management role and 6 worker roles. My Namenode and Master showed not running and I cannot restart the Namenode. Error below:

 

2015-11-29 19:27:32,758 FATAL org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode join
java.io.IOException: Failed to load an FSImage file!
	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:657)
	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:275)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:880)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:639)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:500)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:556)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:721)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:705)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1355)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1421)

 

I ssh'ed into the master role VM and tried to fix it manually:

 

hdfs namenode -recover

But it is still the same. Let me know where I should troubleshoot next. Thanks

 

 

 

 

 

5 REPLIES 5

Re: Cannot restart namenode: Failed to load an FSImage file

Rising Star

Hi hawkphil,

 

What did you actually do in the cluster before you face the error?

Which version are you running on?

 

Dice.

Re: Cannot restart namenode: Failed to load an FSImage file

Explorer

I am on the latest version, whichever I downaloded last week v5.5 and I installed in vSphere

 

My lab got an outage so the entire cluster went off disruptively. That was the main reason causing all weird thing. I didn't have too much valuable data in there yet

Re: Cannot restart namenode: Failed to load an FSImage file

Explorer

Just a gentle nudge...

 

Any suggestion please?

Highlighted

Re: Cannot restart namenode: Failed to load an FSImage file

Expert Contributor
Check if your namenode.name.dir was pointing to a directory where fsimage doesn't exist, make it pointing to the directory where previous fsimage files exit.
Em Jay

Re: Cannot restart namenode: Failed to load an FSImage file

Explorer
It is pointing to /dfs/nn and there is ./current folder in there
I think it's corrupted. I replaced with /dfs/snn/current and it's still not working
Then I use Cloudera Manager to redeploy the Namenode / Secondary Namenode to some other VMs and it seems to work .. for now.
Strange.