Support Questions

Find answers, ask questions, and share your expertise
Announcements
Welcome to the upgraded Community! Read this blog to see What’s New!

HDFS dsquota diskspace consumed calculation

avatar
New Contributor

Hello,

For a directory in HDFS, I have set dsquota as 1GB but when I try to copy a 500 MB file to that directory. I get a exception == copyFromLocal: The DiskSpace quota of /user/prateek is exceeded: quota = 1073741824 B = 1 GB but diskspace consumed = 1305477120 B = 1.22 GB.

The directory already contains four files.

47401-screen-shot-2018-01-01-at-41402-pm.png

1048576 /user/prateek/file1.txt

10485760 /user/prateek/file2.txt

10485760 /user/prateek/file3.txt

10485760 /user/prateek/file4.txt

How can we calculate the diskspace consumed(i.e.1.22GB in this case)?

Thanks in advance

47399-screen-shot-2018-01-01-at-34749-pm.png

1 ACCEPTED SOLUTION

avatar
Rising Star
@Prateek Behera

The disk quota in your case works fine as expected. HDFS by default has a replication factor of 3 as you can see in the 3rd column of your CLI output.

500MB *3 (replication factor) = 1.5 GB > 1GB (quota). 

View solution in original post

1 REPLY 1

avatar
Rising Star
@Prateek Behera

The disk quota in your case works fine as expected. HDFS by default has a replication factor of 3 as you can see in the 3rd column of your CLI output.

500MB *3 (replication factor) = 1.5 GB > 1GB (quota). 
Labels