Answer
New Contributor
Posts: 9
Registered: ‎09-02-2016
Erasure coding on C6

Hi,

 

We set erasure coding policy to RS-6-3-1024k.

 

As document said in http://hadoop.apache.org/docs/r3.1.0/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html

For rack fault-tolerance, it is also important to have at least as many racks as the configured EC stripe width. For EC policy RS (6,3), this means minimally 9 racks, and ideally 10 or 11 to handle planned and unplanned outages. For clusters with fewer racks than the stripe width, HDFS cannot maintain rack fault-tolerance, but will still attempt to spread a striped file across multiple nodes to preserve node-level fault-tolerance.

 

Supposely, we only have 8 racks how come it doesn't have rack fault-tolerance.

rack1 rack2 rack3 rack4 rack5 rack6 rack7 rack8

cell1  cell2   cell3  cell4   cell5  cell6   cell7  cell8

cell9

 

if the rack1 go down cell1 and cell9 can recovery from the other nodes.

what's the case that it doesn't have rack fault-tolerance.

 

Thanks in advance.

 

 

Other Answers: 1
Moderator
Posts: 89
Registered: ‎07-08-2013
Answered
The upstream doc is wrong. We'll file a ticket to get it fixed. The stripe width in the case of RS(6,3) is 6, not 9.