What is the difference between volumes and folders?

ryu — Thu, 16 Dec 2021 00:47:07 GMT

Hi experts,

Can someone please explain the difference between volumes and folders in hadoop?

Thanks,

Re: What is the difference between volumes and folders?

willx — Thu, 16 Dec 2021 01:26:45 GMT

Volume:

As described in HDFS architecture, the NameNode stores metadata while the DataNodes store the actual data content. Each DataNode is a computer which usually consists of multiple disks (in HDFS’ terminology, volumes). A file in HDFS contains one or more blocks. A block has one or multiple copies (called Replicas), based on the configured replication factor. A replica is stored on a volume of a DataNode, and different replicas of the same block are stored on different DataNodes.

https://blog.cloudera.com/hdfs-datanode-scanners-and-disk-checker-explained/

Directory(usually don't say it folders):

like other file system, hdfs directory is hierarchical file structure

https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html

Regards,

Will

Re: What is the difference between volumes and folders?

ryu — Thu, 16 Dec 2021 20:40:18 GMT

Hi @willx ,
Is there a way to see if the hadoop path is a volume or a directory?

question Re: What is the difference between volumes and folders? in Support Questions

What is the difference between volumes and folders?

Re: What is the difference between volumes and folders?

Re: What is the difference between volumes and folders?