- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Why are the HDFS edit logs retained after a checkpoint operation is complete?
- Labels:
-
Apache Hadoop
Created ‎02-10-2016 03:50 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In an HA environment, I see a fairly current fsimage file in the Namenode directory. But I also see edit files for the last 20+ days even though the checkpoint frequency is 6 hours/1M transactions.
Created ‎02-10-2016 04:53 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The hdfs-site.xml property dfs.namenode.num.extra.edits.retained defaults to a value of 1000000 (1 million) Reducing this value will lower the number of edit logs retained after a checkpoint in complete.
Created ‎02-10-2016 03:55 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created ‎02-10-2016 04:50 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Pointed me to the right place to do more research.
Created ‎02-10-2016 04:53 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The hdfs-site.xml property dfs.namenode.num.extra.edits.retained defaults to a value of 1000000 (1 million) Reducing this value will lower the number of edit logs retained after a checkpoint in complete.
