"as per your explanation edits log and fsimage store changes only from the last checkpoint" No. Your HDFS state is stored in FSImage. All the changes (rename, delete, permission change) to last checkpoint will be stored in edit log. To avoid that the edit log to be too large sometimes the SN applies the changes from edit log in FSImage. That means FSImage stores your whole HDFS state not only from the last checkpoint. With other words, the checkpoint is actually the FSImage. This whole process is needed because it is hard to write FSImage it is easier to write the changes into edit log, but edit log could be too large so merge is needed in certain period. Example: You have a file a.txt and you rename to b.txt. After you rename b.txt to c.txt FSImage shows you have a.txt and in edit log it stores you had two renames (a -> b, b -> c) After SN merges FSImage and edit log: Edit log will be empty and FSImage will show you have c.txt. There will be no any record for a.txt. Does it answer you question?
... View more
