Reply
Contributor
Posts: 26
Registered: ‎04-16-2017

Name node recovery mode

Hi,

 

What is the purpose of starting namenode in the recovery mode ?

 

Thanks.

Cloudera Employee
Posts: 41
Registered: ‎10-07-2016

Re: Name node recovery mode

Howdy,

 

Thanks for coming to the community with your question. I'm Josh.

 

We have a pretty extensive engineering blog post outlining namenode recovery tools, one of which is namenode recovery mode.

 

http://blog.cloudera.com/blog/2012/05/namenode-recovery-tools-for-the-hadoop-distributed-file-system...

 

A few code snippets stand out. One is the command used to start a namenode in recovery mode.

"./bin/hadoop namenode -recover"

 

And the other is the text one is greeted with when running the above command

"

You have selected Metadata Recovery mode.  This mode is intended to recover

lost metadata on a corrupt filesystem.  Metadata recovery mode often

permanently deletes data from your HDFS filesystem.  Please back up your edit

log and fsimage before trying this!

"

 

In short, namenode recovery mode checks the edits log for errors, and asks you what you'd like to do about them. 

 

There is one thing I'm curious about. Are you asking this out of curiosity, or do you have an HDFS problem you're trying to solve? Please let me know if you have any other questions or if you'd like further assistance.

 

Cheers,

Josh

 

Contributor
Posts: 26
Registered: ‎04-16-2017

Re: Name node recovery mode

Thanks for your resolution.

 

I am just curious to know more depth about namenode operation. So I asked.

Cloudera Employee
Posts: 41
Registered: ‎10-07-2016

Re: Name node recovery mode

[ Edited ]

I'm glad to be of service. Let me know if there's anything else namenode related that you're curious about. :) 

Contributor
Posts: 26
Registered: ‎04-16-2017

Re: Name node recovery mode

Does Cloudera or any other Hadoop vendor's provide Individual certification in Hive or Pig, instead of Complete Hadoop certification ?

Cloudera Employee
Posts: 41
Registered: ‎10-07-2016

Re: Name node recovery mode

I can't speak for other vendors, but I don't think we offer certifications for single components. Generally, working with hadoop effectively requires familiarity with multiple components, so it behooves oneself to learn a few components in the stack. The closest certifications I can think of are the data engineer and data analyst certifications. There's a link bellow to our full list of certifications for reference. I hope this helps. 

 

https://www.cloudera.com/more/training/certification.html

 

 

Contributor
Posts: 26
Registered: ‎04-16-2017

Re: Name node recovery mode

Thank you so much.

Explorer
Posts: 6
Registered: ‎11-25-2014

Re: Name node recovery mode

I am confused since when I run this job I get

18/10/12 16:14:07 WARN common.Storage: Storage directory /tmp/hadoop-root/dfs/name does not exist

18/10/12 16:14:07 WARN namenode.FSNamesystem: Encountered exception loading fsimage

org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-root/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.

Announcements