We are running a spark streaming job with yarn as resource manager, noticing that these two directories are getting filled up on the data nodes and we are running out of space when we only run for couple of min's
/tmp/hadoop/data/nm-local-dir/filecache
/tmp/hadoop/data/nm-local-dir/filecache
these directories are not getting cleared automatically , from my research found that this property need's to be set, yarn.nodemanager.localizer.cache.cleanup.interval-ms
Even after setting this up ..it's not automatically clearing out any help will be greatly appreciated
<configuration>
~
~ <property>
~ <name>yarn.nodemanager.aux-services</name>
~ <value>mapreduce_shuffle</value>
~ </property>
~
~ <property>
~ <name>yarn.resourcemanager.hostname</name>
~ <value>hdfs-name-node</value>
~ </property>
~
~ <property>
~ <name>yarn.nodemanager.resource.memory-mb</name>
~ <value>16384</value>
~ </property>
~
~ <property>
~ <name>yarn.nodemanager.resource.cpu-vcores</name>
~ <value>6</value>
~ </property>
~
~ <property>
~ <name>yarn.scheduler.maximum-allocation-mb</name>
~ <value>16384</value>
~ </property>
<property>
~ <name>yarn.nodemanager.localizer.cache.cleanup.interval-ms</name>
~ <value>3000</value>
~ </property>
~
~ <!-- Needs to be explicitly set as part of a workaround for YARN-367.
~ | If changing this property, you must also change the
~ | hadoop.tmp.dir property in hdfs-site.xml. This location must always
~ | be a subdirectory of the location specified in hadoop.tmp.dir. This
~ | affects all versions of Yarn 2.0.0 through 2.7.3+. -->
~ <property>
~ <name>yarn.nodemanager.local-dirs</name>
~ <value>file:///tmp/hadoop/data/nm-local-dir</value>
~ </property>
~
~ </configuration>
