Support Questions

Find answers, ask questions, and share your expertise

NameNode edit logs - purging/Best practises

avatar
Explorer

I see edit logs created for every 2 minute interval @"/data/hadoop/hdfs/namenode/current" directory. Is there a best practice to maintain the edit logs in terms of purging.

Also, we have a problem where Namenode fails to start and we use the following post "ttp://blog.cloudera.com/blog/2012/05/namenode-recovery-tools-for-the-hadoop-distributed-file-system/" to recover it. Is there something to do with the edits here.

1 ACCEPTED SOLUTION

avatar

@Bharath N, you don't need to purge edit log files. HDFS deletes them automatically when they are no longer needed. I recommend not touching any edit log files since accidentally deleting the wrong file could lead to data loss.

For your NameNode start problem, I can't say anything without more data. You want to engage Hortonworks support for this problem, if you have a support contract. Else you can post the error message/exception here and we may be able to point you in the right direction.

View solution in original post

2 REPLIES 2

avatar

@Bharath N, you don't need to purge edit log files. HDFS deletes them automatically when they are no longer needed. I recommend not touching any edit log files since accidentally deleting the wrong file could lead to data loss.

For your NameNode start problem, I can't say anything without more data. You want to engage Hortonworks support for this problem, if you have a support contract. Else you can post the error message/exception here and we may be able to point you in the right direction.

avatar
Explorer

Make sure your SecondaryNamenode, or standby Namenode is running it will keep checkpointing and removing old edit logs.