Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Yarn : yarn-site.xml changes not taking effect

Highlighted

Yarn : yarn-site.xml changes not taking effect

Explorer

We have a spark streaming application running on HDFS 2.7.3 with Yarn as the resource manager....while running the application .. these two folders

 

/tmp/hadoop/data/nm-local-dir/filecache

/tmp/hadoop/data/nm-local-dir/filecache

 

are filling up and hence the disk .....

so from my research found that configuring these two properties in yarn-site.xml will help

<property>
<name>yarn.nodemanager.localizer.cache.cleanup.interval-ms</name>
<value>2000</value>
</property>

<property>
<name>yarn.nodemanager.localizer.cache.target-size-mb</name>
<value>2048</value>
</property>

 

 

i have configured them on each namenode and masternode ..and restarted the yarn on masternode ..but still doesn't seem to help ...please let me know if i'm missing anything ...while doing the configuration changes ...this is the yarn-site.xml

 

 

 

<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>hdfs-name-node</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>16384</value>
</property>
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>6</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>16384</value>
</property>
<property>
<name>yarn.nodemanager.local-dirs</name>
<value>file:///tmp/hadoop/data/nm-local-dir</value>
</property>
<property>
<name>yarn.nodemanager.localizer.cache.cleanup.interval-ms</name>
<value>2000</value>
</property>
<property>
<name>yarn.nodemanager.localizer.cache.target-size-mb</name>
<value>2048</value>
</property>
</configuration>

**Update**: i tried with this property but still doesn't clean up and disk is getting full

<property>
<name>yarn.nodemanager.local-dirs</name>
<value>file:///tmp/hadoop/data/nm-local-dir,file:///tmp/hadoop/data/nm-local-dir/filecache,file:///tmp/hadoop/data/nm-local-dir/usercache</value>
</property>

 

1 REPLY 1

Re: Yarn : yarn-site.xml changes not taking effect

Expert Contributor

Hi srirocky.  It's not typical for the distributed cache to fill up the disk, but instead its more typical that log files or intermediate storage of data might.  If the filecache does have lots of files or large files, you may want to check what is included in the distributed cache.  It may be valid or there may be other ways to handle those files.

 

You also mentioned you restarted the resourcemanager, you will also need to restart all the node managers on all workers for settings to take affect.

Don't have an account?
Coming from Hortonworks? Activate your account here