question Re: NameNode edit logs - purging/Best practises in Archives of Support Questions (Read Only)

NameNode edit logs - purging/Best practises

BharathN — Wed, 11 Oct 2017 22:57:18 GMT

I see edit logs created for every 2 minute interval @"/data/hadoop/hdfs/namenode/current" directory. Is there a best practice to maintain the edit logs in terms of purging.

Also, we have a problem where Namenode fails to start and we use the following post "ttp://blog.cloudera.com/blog/2012/05/namenode-recovery-tools-for-the-hadoop-distributed-file-system/" to recover it. Is there something to do with the edits here.

Re: NameNode edit logs - purging/Best practises

ArpitAgarwal — Fri, 13 Oct 2017 05:09:12 GMT

@Bharath N, you don't need to purge edit log files. HDFS deletes them automatically when they are no longer needed. I recommend not touching any edit log files since accidentally deleting the wrong file could lead to data loss.

For your NameNode start problem, I can't say anything without more data. You want to engage Hortonworks support for this problem, if you have a support contract. Else you can post the error message/exception here and we may be able to point you in the right direction.

Re: NameNode edit logs - purging/Best practises

poojan — Mon, 27 Aug 2018 22:29:02 GMT

Make sure your SecondaryNamenode, or standby Namenode is running it will keep checkpointing and removing old edit logs.