question Re: YARN LOCAL DIRS + deletion in Archives of Support Questions (Read Only)

YARN LOCAL DIRS + deletion

mike_bronson7 — Thu, 27 Dec 2018 17:19:03 GMT

hi all

Regarding the data-node machines in our ambari cluster

in YARN --> CONFIG ,

we configured that

 yarn.nodemanager.local-dirs  TO   /var/hadoop/yarn/local

so as we can see blockmgr-XXX folders are created and take a lot space from the local disk ( /var )

is it safe to remove the old blockmgr-XXX folders? ( lets say from 1 day ago )

9433484 /var/hadoop/yarn/local/usercache/airflow/appcache/application_1544689806134_0986/blockmgr-d8628834-137a-4a7b-a833-69a47a4a8d53
9319472 /var/hadoop/yarn/local/usercache/airflow/appcache/application_1544689806134_0986/blockmgr-67632cb1-d552-41b9-9944-31954964ad9c
8588556 /var/hadoop/yarn/local/usercache/airflow/appcache/application_1544689806134_0986/blockmgr-d159be18-ca81-4974-9248-e79ac5db7981
7622108 /var/hadoop/yarn/local/usercache/airflow/appcache/application_1544689806134_0986/blockmgr-766b5a33-8478-4102-9457-47535ba57ddc

Re: YARN LOCAL DIRS + deletion

jagadeesan — Thu, 27 Dec 2018 17:33:40 GMT

@Michael Bronson

Yes you can remove old files. Below article help you to with proper steps.

https://community.hortonworks.com/articles/92339/how-to-clear-local-file-cache-and-user-cache-for-y.html

Please accept the answer you found most useful.

Re: YARN LOCAL DIRS + deletion

mike_bronson7 — Thu, 27 Dec 2018 17:48:29 GMT

@Jagadeesan A S

can you refer also to my question on the link - https://community.hortonworks.com/questions/232062/yarn-local-dirs-safety-moving-the-local-dir-to-dat.html

Re: YARN LOCAL DIRS + deletion

jagadeesan — Thu, 27 Dec 2018 18:25:19 GMT

@Michael Bronson Sure