Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

what is meaning of datanode numBlocksFailedToUncache metric

avatar
Explorer

Hello,

 

I am seeing this value for numBlocksFailedToUncache property is going up. Should I be concerned? From apache documentation I can see it means https://blog.woopi.org/wordpress/files/hadoop-2.6.0-javadoc/org/apache/hadoop/hdfs/server/datanode/f...

"Number of uncache commands that could not be completed successfully" Not sure how to interpret this..

 

Thanks

M

1 ACCEPTED SOLUTION

avatar
Expert Contributor

Hello

 

This is related to the Cache Management of HDFS

 

As described in the documentation: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.ht...

 

In this architecture, the NameNode is responsible for coordinating all the DataNode off-heap caches in the cluster. The NameNode periodically receives a cache report from each DataNode which describes all the blocks cached on a given DN. The NameNode manages DataNode caches by piggybacking cache and uncache commands on the DataNode heartbeat.

 

If the metric is going up, one possibility could be your namenode is too busy to handle the request

 

 

 

View solution in original post

1 REPLY 1

avatar
Expert Contributor

Hello

 

This is related to the Cache Management of HDFS

 

As described in the documentation: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.ht...

 

In this architecture, the NameNode is responsible for coordinating all the DataNode off-heap caches in the cluster. The NameNode periodically receives a cache report from each DataNode which describes all the blocks cached on a given DN. The NameNode manages DataNode caches by piggybacking cache and uncache commands on the DataNode heartbeat.

 

If the metric is going up, one possibility could be your namenode is too busy to handle the request