Support Questions
Find answers, ask questions, and share your expertise

How file write happens in isilon with hadoop? How is the file to be written split?

Highlighted

How file write happens in isilon with hadoop? How is the file to be written split?

 
3 REPLIES 3

Re: How file write happens in isilon with hadoop? How is the file to be written split?

Isilon uses it own OneFS filesystem to read and write files.

The write process follows a certain flow:

•First, the Client sends file data as a write, which is received by the write coalescer on the client-connected node.

•Then, the node breaks the file in up to 128 KB stripe units and a write plan determines how to aggregate the data stripe units and calculates protection stripe units.

•Finally, stripe units are sent to participant nodes over the InfiniBand network to distribute the stripe and a write occur on each node.

Highlighted

Re: How file write happens in isilon with hadoop? How is the file to be written split?

@Nicholas what is protection stripe unit? is the data divided into stripes of 128KB each or 8KB each? If 8KB, doesn't this cause performance overhead?

Highlighted

Re: How file write happens in isilon with hadoop? How is the file to be written split?

128KB stripes