question Re: about HDFS-7097 in Archives of Support Questions (Read Only)

about HDFS-7097

fatkun — Mon, 04 May 2015 13:11:32 GMT

I found a patch in https://issues.apache.org/jira/browse/HDFS-7097

The author says : “Since block reports are not modifying any state that is being saved to fsimage, I propose letting them through during checkpointing.”

block reports will change block map, this will save in fsimage also.

why do not need lock block reports? how fsimage , edit log and block reports combin to a file?

Re: about HDFS-7097

Harsh J — Mon, 04 May 2015 13:28:13 GMT

The block mapping of block ID <=> DN location is not stored in the
fsimage of HDFS, it is just kept in the running NameNode memory.

Since the primary work of the block report is to report statuses of
availability of each block replica on the DNs, and the checkpoint's
work is to persist the namespace and namespace-associative states
(such as snapshots, etc.), the checkpoint can be an operation done
without problems in parallel as their two information structures will
not conflict with one another.