Reply
Highlighted
New Contributor
Posts: 2
Registered: ‎11-09-2015
Accepted Solution

Changing YARN local directory safely

Hi all.

 

Scenario: Yarn has two local directories on different partitions. One of those partitions has filled due to me undersestimating how much disk yarn would need. The NM is now in a critical health state even though the second partition has a pile of free space.

 

Can I just remove the first partition from the config and restart the service to force it to use the other without breaking stuff?

Posts: 1,892
Kudos: 432
Solutions: 302
Registered: ‎07-31-2013

Re: Changing YARN local directory safely

A YARN NM does not persist any vital data. While it does persist some running container states (for restart recovery purposes), most of these containers can be retried by their applications (such as MR2 and Spark) so their transient data directories are not important to keep across restarts.

> Can I just remove the first partition from the config and restart the service to force it to use the other without breaking stuff?

Yes, this would be safe to perform.
New Contributor
Posts: 2
Registered: ‎11-09-2015

Re: Changing YARN local directory safely

Many thanks! I will perform this operation.
Announcements