Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Why do we need HDFS layer on top of OS layer ?

Why do we need HDFS layer on top of OS layer ?

New Contributor

Why do we need HDFS on top of OS filesystem Layer ?

 

HDFS store files in 128MB blocks whereas these HDFS files when it comes to OS storage , Its stored as 4k blocks or 8k blocks

So If we could Implement the same logic in the OS layer , 

 

Can this be faster than HDFS ?

 

 

1 REPLY 1

Re: Why do we need HDFS layer on top of OS layer ?

Master Guru
HDFS block size is logical. It is not a "pre-allocated" block. The
block division is for benefit of processing and availability, chiefly.

HDFS is also more than just a block based file-system, it is primarily
a distributed file-system. How do you expect to achieve a common
namespace over several "just OS" nodes?