Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Name node recovery mode

Name node recovery mode

Explorer

Hi,

 

What is the purpose of starting namenode in the recovery mode ?

 

Thanks.

7 REPLIES 7

Re: Name node recovery mode

Rising Star

Howdy,

 

Thanks for coming to the community with your question. I'm Josh.

 

We have a pretty extensive engineering blog post outlining namenode recovery tools, one of which is namenode recovery mode.

 

http://blog.cloudera.com/blog/2012/05/namenode-recovery-tools-for-the-hadoop-distributed-file-system...

 

A few code snippets stand out. One is the command used to start a namenode in recovery mode.

"./bin/hadoop namenode -recover"

 

And the other is the text one is greeted with when running the above command

"

You have selected Metadata Recovery mode.  This mode is intended to recover

lost metadata on a corrupt filesystem.  Metadata recovery mode often

permanently deletes data from your HDFS filesystem.  Please back up your edit

log and fsimage before trying this!

"

 

In short, namenode recovery mode checks the edits log for errors, and asks you what you'd like to do about them. 

 

There is one thing I'm curious about. Are you asking this out of curiosity, or do you have an HDFS problem you're trying to solve? Please let me know if you have any other questions or if you'd like further assistance.

 

Cheers,

Josh

 

Re: Name node recovery mode

Explorer

Thanks for your resolution.

 

I am just curious to know more depth about namenode operation. So I asked.

Highlighted

Re: Name node recovery mode

Rising Star

I'm glad to be of service. Let me know if there's anything else namenode related that you're curious about. :) 

Re: Name node recovery mode

Explorer

Does Cloudera or any other Hadoop vendor's provide Individual certification in Hive or Pig, instead of Complete Hadoop certification ?

Re: Name node recovery mode

Rising Star

I can't speak for other vendors, but I don't think we offer certifications for single components. Generally, working with hadoop effectively requires familiarity with multiple components, so it behooves oneself to learn a few components in the stack. The closest certifications I can think of are the data engineer and data analyst certifications. There's a link bellow to our full list of certifications for reference. I hope this helps. 

 

https://www.cloudera.com/more/training/certification.html

 

 

Re: Name node recovery mode

Explorer

Thank you so much.

Re: Name node recovery mode

Explorer

I am confused since when I run this job I get

18/10/12 16:14:07 WARN common.Storage: Storage directory /tmp/hadoop-root/dfs/name does not exist

18/10/12 16:14:07 WARN namenode.FSNamesystem: Encountered exception loading fsimage

org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-root/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.