Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 09-04-2017 06:08 PM
Hello everyone,
From what I understood every files use a minimum of 1 block in HDFS.
So how can it be possible to have more files than blocks?
Below is an exemple:
$ hdfs fsck /app-logs Total size: 14663835874 B Total dirs: 2730 Total files: 8694 Total symlinks: 0 Total blocks (validated): 8690 (avg. block size 1687437 B) Minimally replicated blocks: 8690 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 3 Average block replication: 3.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 4 Number of racks: 1 FSCK ended at Mon Sep 04 19:54:45 CEST 2017 in 353 milliseconds The filesystem under path '/app-logs' is HEALTHY
I appreciate any help in understanding this ouput.
Regards
Created 09-08-2017 07:55 PM
bq. From what I understood every files use a minimum of 1 block in HDFS.
No. It is not true. You can have file of size 0, they are likely to be created on NN but no block being allocated and/or streamed to DNs yet. Check your files and you will have 4 files (8694-8690=4) of size 0.
Created 09-08-2017 07:55 PM
bq. From what I understood every files use a minimum of 1 block in HDFS.
No. It is not true. You can have file of size 0, they are likely to be created on NN but no block being allocated and/or streamed to DNs yet. Check your files and you will have 4 files (8694-8690=4) of size 0.
Created 09-11-2017 07:00 AM
Thank you @Xiaoyu Yao.