Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

What happens exactly when decommisioning a workner node?

avatar

During the admin class two interesting questions on decommissioning worker nodes emerged:

- How does the decommissioning process stops the ApplicationMaster? Will it let the containers complete its current tasks or will it stop the container and recover the task on a different worker node? I assumed the last, but I would like to be sure.

- Will a decommissioned node still hold its physical data when being recommissioned? When is it removed? Does the removal occur at the first rebalancing when you initially decommission the node or will the data blocks still physically exist when you recommission the worker node?

Many thanks to clarify this!

1 ACCEPTED SOLUTION

avatar
Expert Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
2 REPLIES 2

avatar
Expert Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Expert Contributor

Does physical data really remain in the node?

For my case I saw lots of (thousands and repeating) HDFS log lines (on the node) about deleting blocks.

Are these lines unexpected (already have broken RAID controller)?

They keep being scheduled and deleted.

2018-06-14 11:58:53,005 INFO impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:run(308)) - Deleted BP-1789482724-9.1.10.22-1491814552298 blk_1155905937_82210162 file /grid/2/hadoop/hdfs/data/current/BP-1789482724-9.1.10.22-1491814552298/current/finalized/subdir229/subdir185/blk_1155905937