Posts: 46
Registered: ‎11-03-2014
Accepted Solution

After server crash, HA Standby NameNode "Premature EOF from inputStream" ; JournalNode out of sync

CDH 5.1.3 installed with Parcel, HDFS HA enabled.


After a server crash (node running NameNode in standby mode and JournalNode), issues occur during restart. 


In NameNode log:

Failed to load image from FSImageFile(file=/data1/dfs/nn/current/fsimage_0000000000004637167, cpktTxId=0000000000004637167) Premature EOF from inputStream
at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(
at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(


In JournalNode log:

IPC Server handler 3 on 8485, call org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocol.getEditLogManifest from Call#2 Retry#0: output error


How can I recover the node?



Posts: 46
Registered: ‎11-03-2014

Re: After server crash, HA Standby NameNode "Premature EOF from inputStream" ; JournalNode

Not sure whether all these are correct. But I managed to make all instance "Green" with some manual operation...



  • I stopped another (good) JournalNode and copied the 'jn/[nameservice]/current' directory to the bad JournalNode. I tried copying while the good JournalNode was running, but starting the bad JournalNode gave same error as previously.


  • I copied the fsimage file mentioned by the log from the a good NameNode.