Support Questions
Find answers, ask questions, and share your expertise

no namenode to stop when stop namenode

when we stop the name-node as the following , then we get - no namenode to stop

what this happens ? , and what is the resolution for that ?

su hdfs -l -s /bin/bash -c 'ulimit -c unlimited ;  /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config /usr/hdp/current/hadoop-client/conf stop namenode' 



no namenode to stop
Michael-Bronson
13 REPLIES 13

Re: no namenode to stop when stop namenode

@Michael Bronson

Looks like Namenode is not running in this machine. check if the process is running using "ps -ef | grep namenode"

Re: no namenode to stop when stop namenode

the problem is we cant start the namenode , so I also try to restrat it as stop/start , but the main problem is that we cant start the name node

Michael-Bronson

Re: no namenode to stop when stop namenode

from the log we can see -

2018-01-24 20:20:56,489 ERROR namenode.NameNode (NameNode.java:main(1774)) - Failed to start namenode.
java.io.FileNotFoundException: No valid image files found

when we start the name node , do you can advice how to resolve this ? 
Michael-Bronson

Re: no namenode to stop when stop namenode

I think we need to put the focus on - "No valid image files found: what this is mean ?

Michael-Bronson

Re: no namenode to stop when stop namenode

I am also try this - hdfs namenode -recover ( but without help )

WARN namenode.FSNamesystem: Encountered exception loading fsimage
java.io.FileNotFoundException: No valid image files found
        at org.apache.hadoop.hdfs.server.namenode.FSImageTransactionalStorageInspector.getLatestImages(FSImageTransactionalStorageInspector.java:165)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:618)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:289)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1045)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:703)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1580)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1674)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1769)
18/01/24 20:44:40 INFO namenode.MetaRecoveryContext: RECOVERY FAILED: caught exception
java.io.FileNotFoundException: No valid image files found
        at org.apache.hadoop.hdfs.server.namenode.FSImageTransactionalStorageInspector.getLatestImages(FSImageTransactionalStorageInspector.java:165)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:618)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:289)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1045)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:703)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1580)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1674)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1769)
18/01/24 20:44:40 ERROR namenode.NameNode: Failed to start namenode.
java.io.FileNotFoundException: No valid image files found
        at org.apache.hadoop.hdfs.server.namenode.FSImageTransactionalStorageInspector.getLatestImages(FSImageTransactionalStorageInspector.java:165)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:618)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:289)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1045)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:703)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.doRecovery(NameNode.java:1580)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1674)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1769)
Michael-Bronson

Re: no namenode to stop when stop namenode

Is this a new cluster? if not was there any change when this issue started occured.

Re: no namenode to stop when stop namenode

this is old cluster , many hands works on that cluster

Michael-Bronson

Re: no namenode to stop when stop namenode

The above exception says that you don't have a fsimage in the "dfs.namenode.name.dir" . Is NN HA enabled? if yes is another NN working as expected. if so you could run a "hdfs namenode -bootstrapStandby" on Standby NameNode and then start the NN.

Re: no namenode to stop when stop namenode

we run it on standby machine and we got -

18/01/24 22:29:21 INFO ipc.Client: Retrying connect to server: master01.sys57.com/100.14.22.27:8020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
Michael-Bronson