Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Changing YARN local directory safely

avatar
New Contributor

Hi all.

 

Scenario: Yarn has two local directories on different partitions. One of those partitions has filled due to me undersestimating how much disk yarn would need. The NM is now in a critical health state even though the second partition has a pile of free space.

 

Can I just remove the first partition from the config and restart the service to force it to use the other without breaking stuff?

1 ACCEPTED SOLUTION

avatar
Mentor
A YARN NM does not persist any vital data. While it does persist some running container states (for restart recovery purposes), most of these containers can be retried by their applications (such as MR2 and Spark) so their transient data directories are not important to keep across restarts.

> Can I just remove the first partition from the config and restart the service to force it to use the other without breaking stuff?

Yes, this would be safe to perform.

View solution in original post

2 REPLIES 2

avatar
Mentor
A YARN NM does not persist any vital data. While it does persist some running container states (for restart recovery purposes), most of these containers can be retried by their applications (such as MR2 and Spark) so their transient data directories are not important to keep across restarts.

> Can I just remove the first partition from the config and restart the service to force it to use the other without breaking stuff?

Yes, this would be safe to perform.

avatar
New Contributor
Many thanks! I will perform this operation.