Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

about HDFS-7097

avatar
Explorer

I found a patch in https://issues.apache.org/jira/browse/HDFS-7097

 

The author says : “Since block reports are not modifying any state that is being saved to fsimage, I propose letting them through during checkpointing.

 

block reports will change block map, this will save in fsimage also.

 

why do not need lock block reports? how fsimage , edit log and block reports combin to a file?

1 ACCEPTED SOLUTION

avatar
Mentor
The block mapping of block ID <=> DN location is not stored in the
fsimage of HDFS, it is just kept in the running NameNode memory.

Since the primary work of the block report is to report statuses of
availability of each block replica on the DNs, and the checkpoint's
work is to persist the namespace and namespace-associative states
(such as snapshots, etc.), the checkpoint can be an operation done
without problems in parallel as their two information structures will
not conflict with one another.

View solution in original post

1 REPLY 1

avatar
Mentor
The block mapping of block ID <=> DN location is not stored in the
fsimage of HDFS, it is just kept in the running NameNode memory.

Since the primary work of the block report is to report statuses of
availability of each block replica on the DNs, and the checkpoint's
work is to persist the namespace and namespace-associative states
(such as snapshots, etc.), the checkpoint can be an operation done
without problems in parallel as their two information structures will
not conflict with one another.