YARN LOCAL DIRS + deletion

[#1]

hi all

Regarding the data-node machines in our ambari cluster

in YARN --> CONFIG ,

we configured that

 yarn.nodemanager.local-dirs  TO   /var/hadoop/yarn/local  

so as we can see blockmgr-XXX folders are created and take a lot space from the local disk ( /var )

is it safe to remove the old blockmgr-XXX folders? ( lets say from 1 day ago )

9433484 /var/hadoop/yarn/local/usercache/airflow/appcache/application_1544689806134_0986/blockmgr-d8628834-137a-4a7b-a833-69a47a4a8d53
9319472 /var/hadoop/yarn/local/usercache/airflow/appcache/application_1544689806134_0986/blockmgr-67632cb1-d552-41b9-9944-31954964ad9c
8588556 /var/hadoop/yarn/local/usercache/airflow/appcache/application_1544689806134_0986/blockmgr-d159be18-ca81-4974-9248-e79ac5db7981
7622108 /var/hadoop/yarn/local/usercache/airflow/appcache/application_1544689806134_0986/blockmgr-766b5a33-8478-4102-9457-47535ba57ddc

[#2 · Accepted solution]

@Michael Bronson

Yes you can remove old files. Below article help you to with proper steps.

https://community.hortonworks.com/articles/92339/how-to-clear-local-file-cache-and-user-cache-for-y....

Please accept the answer you found most useful.

[#3]

@Jagadeesan A S

can you refer also to my question on the link - https://community.hortonworks.com/questions/232062/yarn-local-dirs-safety-moving-the-local-dir-to-da...

[#4]

@Michael Bronson Sure