Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

about HDFS-7097

avatar
Explorer

I found a patch in https://issues.apache.org/jira/browse/HDFS-7097

 

The author says : “Since block reports are not modifying any state that is being saved to fsimage, I propose letting them through during checkpointing.

 

block reports will change block map, this will save in fsimage also.

 

why do not need lock block reports? how fsimage , edit log and block reports combin to a file?

1 ACCEPTED SOLUTION

avatar
Mentor
The block mapping of block ID <=> DN location is not stored in the
fsimage of HDFS, it is just kept in the running NameNode memory.

Since the primary work of the block report is to report statuses of
availability of each block replica on the DNs, and the checkpoint's
work is to persist the namespace and namespace-associative states
(such as snapshots, etc.), the checkpoint can be an operation done
without problems in parallel as their two information structures will
not conflict with one another.

View solution in original post

1 REPLY 1

avatar
Mentor
The block mapping of block ID <=> DN location is not stored in the
fsimage of HDFS, it is just kept in the running NameNode memory.

Since the primary work of the block report is to report statuses of
availability of each block replica on the DNs, and the checkpoint's
work is to persist the namespace and namespace-associative states
(such as snapshots, etc.), the checkpoint can be an operation done
without problems in parallel as their two information structures will
not conflict with one another.