Hi @wert_1311 , Thanks for asking. Currently yarn.nodemanager.localizer.cache.target-size-mb and yarn.nodemanager.localizer.cache.cleanup.interval-ms triggers deletion service for non-running containers. However, for containers that are running and spilling data to {'yarn.nodemanager.local-dirs'}/usercache/<user>/appcache/<app_id>, the deletion service does not come into action, as a result, filesystem gets full, nodes are marked unhealthy and application gets stuck. I suggest you refer to an internal community article [1] which speaks about something similar. I think that the upstream JIRA [YARN-4540] [2] has this documented and is yet to be unresolved. The general recommendation is to just make that FS big and if it gets full, debug the job that writes too much data into it. Also, It is ok about deleting the usercache dir. Use the following steps to delete the usercache: Stop the YARN service. Log in to all nodes and delete the content of the usercache directories. For example: for i in `cat list_of_nodes_in_cluster`; do ssh $i rm -rf /data?/yarn/nm/usercache/* ; done Verify all usercache directories on all nodes are empty. Start the YARN service. Please let us know if this is helpful. [1] https://community.cloudera.com/t5/Support-Questions/yarn-usercache-folder-became-with-huge-size/td-p/178648 [2] https://issues.apache.org/jira/browse/YARN-4540
... View more
