Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How file write happens in isilon with hadoop? How is the file to be written split?

Highlighted

How file write happens in isilon with hadoop? How is the file to be written split?

New Contributor
 
3 REPLIES 3

Re: How file write happens in isilon with hadoop? How is the file to be written split?

New Contributor

Isilon uses it own OneFS filesystem to read and write files.

The write process follows a certain flow:

•First, the Client sends file data as a write, which is received by the write coalescer on the client-connected node.

•Then, the node breaks the file in up to 128 KB stripe units and a write plan determines how to aggregate the data stripe units and calculates protection stripe units.

•Finally, stripe units are sent to participant nodes over the InfiniBand network to distribute the stripe and a write occur on each node.

Re: How file write happens in isilon with hadoop? How is the file to be written split?

New Contributor

@Nicholas what is protection stripe unit? is the data divided into stripes of 128KB each or 8KB each? If 8KB, doesn't this cause performance overhead?

Re: How file write happens in isilon with hadoop? How is the file to be written split?

New Contributor

128KB stripes