Skip to main content
SDL
SDL
Silver ⭐️⭐️
March 29, 2021
Solved

what is meaning of datanode numBlocksFailedToUncache metric

Hello,

 

I am seeing this value for numBlocksFailedToUncache property is going up. Should I be concerned? From apache documentation I can see it means https://blog.woopi.org/wordpress/files/hadoop-2.6.0-javadoc/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetCache.html

"Number of uncache commands that could not be completed successfully" Not sure how to interpret this..

 

Thanks

M

Daming Xue
Daming XueAnswer
Gold ⭐️
March 30, 2021

Hello

 

This is related to the Cache Management of HDFS

 

As described in the documentation: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html

 

In this architecture, the NameNode is responsible for coordinating all the DataNode off-heap caches in the cluster. The NameNode periodically receives a cache report from each DataNode which describes all the blocks cached on a given DN. The NameNode manages DataNode caches by piggybacking cache and uncache commands on the DataNode heartbeat.

 

If the metric is going up, one possibility could be your namenode is too busy to handle the request

 

 

 

View in thread

1 reply

Daming Xue
Gold ⭐️
Daming XueAnswer
Gold ⭐️
March 30, 2021

Hello

 

This is related to the Cache Management of HDFS

 

As described in the documentation: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html

 

In this architecture, the NameNode is responsible for coordinating all the DataNode off-heap caches in the cluster. The NameNode periodically receives a cache report from each DataNode which describes all the blocks cached on a given DN. The NameNode manages DataNode caches by piggybacking cache and uncache commands on the DataNode heartbeat.

 

If the metric is going up, one possibility could be your namenode is too busy to handle the request

 

 

 

Translate
    Terms & ConditionsAccessibility statement