BlockCapacity given in the REST reponse of Namenode is different than what we have set in the Ambari under HDFS dfs.blocksize
On Rest I am getting the value for BlockCapacity is 2097152 which equivalent to 2mb although the value of blocksize in Ambari is 128mb by default Fetching the blockcapacity from from following REST url https://<namenode-ip>:50470/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem
Does BlockCapacity from namenodes rest api and dfs.blocksize from Ambari are the same or different?
Created on 03-16-2020 06:41 AM - edited 03-16-2020 06:41 AM
@MAS The BlockCapacity metric is different from Blocksize. Per the metrics page [https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Metrics.html], block capacity denotes "Current number of block capacity" its unit is not bytes/mb it is number of blocks
Created 03-20-2020 01:50 AM
@venkatsambath If it is the value of blocks, then how 2097152 blocks can be fit in 100gb space cluster each of 128 mb of block? Acutally i'm not getting how this block capacity value will be calculated, while calculating this metric total memory is considered or not?
I have single node hadoop cluster as well as multinode test envirnonment in both the value for block capacity is same 2097152.
Where should i found the block size in rest response? Is there any other URI for this metric? Initially it was assumed that the block capacity is same block size but over the time this value got changed.
Created 03-22-2020 05:42 AM
The capacity is not determined based on your cluster size rather estimated based on NN memory. Refer the code here for the computation details https://github.com/cloudera/hadoop-common/blob/134a29f22676e9a5226a07de6c6b9f34934a4625/hadoop-commo...
BlockSize details are not available in rest response of NN. But you may use RestAPI on ClouderaManager to get the configs. Example is in link: https://docs.cloudera.com/documentation/enterprise/5-14-x/topics/cm_intro_api.html#xd_583c10bfdbd326...
For Ambari check in link https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md