Skip to main content
mig_aguir
mig_aguir
Gold ⭐️
January 9, 2021
Solved

Remove directory from yarn.nodemanager.local-dirs - CDH 6.3

We made a mistake while configuring the storage of our worker nodes and we ended up adding a data mount point of a really small disk to the yarn.nodemanager.local-dirs property. This causes some of our very data-intensive spark jobs to fail as this disk runs out of space pretty quickly as these jobs generate a lot of staging data. Right now the value of this property is the following:

 

/u01/hadoop/yarn/nm,/u02/hadoop/yarn/nm,/u03/hadoop/yarn/nm,/u04/hadoop/yarn/nm

 

We just want to remove /u04/hadoop/yarn/nm from the property, all the other directories are mounted on disks with enough capacity.

So, is it safe to just remove this directory from the yarn.nodemanager.local-dirs property? Or do we have to back up the data in /u04/hadoop/yarn/nm and create another data mount point using a disk with enough capacity and put that backup there?

GangWar
GangWarAnswer
Platinum ⭐️⭐️
January 9, 2021

@mig_aguir From the given yarn-site.xml file "yarn.nodemanager.local-dirs" You can change above property to point to new mount directory and post restart of Yarn service new application job will use this directory for storing. 

 

For safe side you can take a backup and store locally for time being if something goes wrong. 

Cheers! Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
View in thread

3 replies

GangWar
Platinum ⭐️⭐️
GangWarAnswer
Platinum ⭐️⭐️
January 9, 2021

@mig_aguir From the given yarn-site.xml file "yarn.nodemanager.local-dirs" You can change above property to point to new mount directory and post restart of Yarn service new application job will use this directory for storing. 

 

For safe side you can take a backup and store locally for time being if something goes wrong. 

Translate
Cheers! Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
mig_aguir
mig_aguirAuthor
Gold ⭐️
January 9, 2021

Cool, thanks for the help! Just another question, if I just were to remove the directory I don't want to use for storing staging data and then restarted Yarn service, would there be any problem? The "yarn.nodemanager.local-dirs" property already includes three other directories for storing so it shouldn't be a problem, right?

Translate
GangWar
Platinum ⭐️⭐️
GangWar
Platinum ⭐️⭐️
January 10, 2021

@mig_aguir Yes, I agree with you. 

Translate
Cheers! Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
Terms & ConditionsAccessibility statement