01-13-2018 12:08 PM
Hi! I'd like to ask what is the actual meaning of the 'Block Capacity' chart for HDFS. The tsquery expression is known: 'SELECT block_capacity ...', but can't figure out where block_capacity is retrived from or how it is calculated.
The question arose after we have increased the 'Java Heap Size of NameNode in Bytes' for our Namenodes - we'd expect the block_capacity of the cluster would also grow, but actually it remains the same. Maybe we'd need to restart cluster to get this parameter updated?
01-18-2018 01:39 PM
As far as I understand, Block Capacity means the total number of blocks HDFS can hold, irrespective of the size. For example, a file of 128MB size will consume 1 HDFS block (assuming HDFS block size is set to 128MB) from a Data Node perspective, but on the NameNode, it needs 2 namespace objects (1 for file inode and 1 block).
Since all that is stored in memory, the block capacity should increave after increasing the heap size of namenode. Yes, you will have to restart HDFS and dependent services to see the increased capacity. However, it might take some time for it to reflect...