https://community.cloudera.com/t5/Support-Questions/yarn-usercache-size-of-local-folder-increasing-in-spite-yarn/m-p/241018#M202822 <P>hi all</P><P>we noticed about the following problem:</P><P>under the folder ( DATANODE machine )</P><PRE>/var/hadoop/yarn/local/usercache</PRE><P><STRONG>size is 140G </STRONG></P><P>in spite we set the following configuration that should avoid this case</P><PRE>yarn.nodemanager.localizer.cache.target-size-mb = 10240 yarn.nodemanager.localizer.cache.cleanup.interval-ms = 300000</PRE><P>this configuration from my understanding should delete the the folder/files in case size is more then 10G</P><P>so how it can be current size is 140G?</P><P>reference:</P><P><A href="https://community.hortonworks.com/questions/201820/yarn-usercache-folder-became-with-huge-size.html" target="_blank">https://community.hortonworks.com/questions/201820/yarn-usercache-folder-became-with-huge-size.html</A></P><UL> <LI><STRONG>yarn.nodemanager.localizer.cache.target-size-mb</STRONG>: This decides the maximum disk space to be used for localizing resources. (At present there is no individual limit for PRIVATE / APPLICATION / PUBLIC cache. <A href="https://issues.apache.org/jira/browse/YARN-882">YARN-882</A>). Once the total disk size of the cache exceeds this then Deletion service will try to remove files which are not used by any running containers. At present there is no limit (quota) for user cache / public cache / private cache. This limit is applicable to all the disks as a total and is not based on per disk basis.</LI><LI><STRONG>yarn.nodemanager.localizer.cache.cleanup.interval-ms</STRONG>: After this interval resource localization service will try to delete the unused resources if total cache size exceeds the configured max-size. Unused resources are those resources which are not referenced by any running container. Every time container requests a resource, container is added into the resources’ reference list. It will remain there until container finishes avoiding accidental deletion of this resource. As a part of container resource cleanup (when container finishes) container will be removed from resources’ reference list. That is why when reference count drops to zero it is an ideal candidate for deletion. The resources will be deleted on LRU basis until current cache size drops below target size.</LI></UL> Tue, 25 Dec 2018 19:09:16 GMT mike_bronson7 2018-12-25T19:09:16Z https://community.cloudera.com/t5/Support-Questions/yarn-usercache-size-of-local-folder-increasing-in-spite-yarn/m-p/241018#M202822 <P>hi all</P><P>we noticed about the following problem:</P><P>under the folder ( DATANODE machine )</P><PRE>/var/hadoop/yarn/local/usercache</PRE><P><STRONG>size is 140G </STRONG></P><P>in spite we set the following configuration that should avoid this case</P><PRE>yarn.nodemanager.localizer.cache.target-size-mb = 10240 yarn.nodemanager.localizer.cache.cleanup.interval-ms = 300000</PRE><P>this configuration from my understanding should delete the the folder/files in case size is more then 10G</P><P>so how it can be current size is 140G?</P><P>reference:</P><P><A href="https://community.hortonworks.com/questions/201820/yarn-usercache-folder-became-with-huge-size.html" target="_blank">https://community.hortonworks.com/questions/201820/yarn-usercache-folder-became-with-huge-size.html</A></P><UL> <LI><STRONG>yarn.nodemanager.localizer.cache.target-size-mb</STRONG>: This decides the maximum disk space to be used for localizing resources. (At present there is no individual limit for PRIVATE / APPLICATION / PUBLIC cache. <A href="https://issues.apache.org/jira/browse/YARN-882">YARN-882</A>). Once the total disk size of the cache exceeds this then Deletion service will try to remove files which are not used by any running containers. At present there is no limit (quota) for user cache / public cache / private cache. This limit is applicable to all the disks as a total and is not based on per disk basis.</LI><LI><STRONG>yarn.nodemanager.localizer.cache.cleanup.interval-ms</STRONG>: After this interval resource localization service will try to delete the unused resources if total cache size exceeds the configured max-size. Unused resources are those resources which are not referenced by any running container. Every time container requests a resource, container is added into the resources’ reference list. It will remain there until container finishes avoiding accidental deletion of this resource. As a part of container resource cleanup (when container finishes) container will be removed from resources’ reference list. That is why when reference count drops to zero it is an ideal candidate for deletion. The resources will be deleted on LRU basis until current cache size drops below target size.</LI></UL> Tue, 25 Dec 2018 19:09:16 GMT https://community.cloudera.com/t5/Support-Questions/yarn-usercache-size-of-local-folder-increasing-in-spite-yarn/m-p/241018#M202822 mike_bronson7 2018-12-25T19:09:16Z https://community.cloudera.com/t5/Support-Questions/yarn-usercache-size-of-local-folder-increasing-in-spite-yarn/m-p/241019#M202823 <P>@<A href="https://community.hortonworks.com/users/26229/uribarih.html">Michael Bronson</A></P><P>As for automatic clean up not getting triggered, well it may be due to (or at-least related to) this unresolved bug reported on Yarn.<BR /><A href="https://issues.apache.org/jira/browse/YARN-4540" target="_blank">https://issues.apache.org/jira/browse/YARN-4540</A></P> Wed, 26 Dec 2018 14:02:39 GMT https://community.cloudera.com/t5/Support-Questions/yarn-usercache-size-of-local-folder-increasing-in-spite-yarn/m-p/241019#M202823 jagadeesan 2018-12-26T14:02:39Z