Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What happens exactly when decommisioning a workner node?

Solved Go to solution

What happens exactly when decommisioning a workner node?

New Contributor

During the admin class two interesting questions on decommissioning worker nodes emerged:

- How does the decommissioning process stops the ApplicationMaster? Will it let the containers complete its current tasks or will it stop the container and recover the task on a different worker node? I assumed the last, but I would like to be sure.

- Will a decommissioned node still hold its physical data when being recommissioned? When is it removed? Does the removal occur at the first rebalancing when you initially decommission the node or will the data blocks still physically exist when you recommission the worker node?

Many thanks to clarify this!

1 ACCEPTED SOLUTION

Accepted Solutions

Re: What happens exactly when decommisioning a workner node?

Rising Star

@Ludovic Janssens

Please refer to the following doc :

https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.1/bk_Ambari_Users_Guide/content/_decommission...

and

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_Sys_Admin_Guides/content/ref-b50b4ee6-0d...

to understand #1

To answer #2, Yes the physical data will remain in the worker node (unless you delete the node). You will need to rebalance once you recommission your node.

Refer point #7 here :

https://acadgild.com/blog/commissioning-and-decommissioning-of-datanode-in-hadoop/

Hope this helps!

2 REPLIES 2

Re: What happens exactly when decommisioning a workner node?

Rising Star

@Ludovic Janssens

Please refer to the following doc :

https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.1/bk_Ambari_Users_Guide/content/_decommission...

and

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_Sys_Admin_Guides/content/ref-b50b4ee6-0d...

to understand #1

To answer #2, Yes the physical data will remain in the worker node (unless you delete the node). You will need to rebalance once you recommission your node.

Refer point #7 here :

https://acadgild.com/blog/commissioning-and-decommissioning-of-datanode-in-hadoop/

Hope this helps!

Re: What happens exactly when decommisioning a workner node?

Rising Star

Does physical data really remain in the node?

For my case I saw lots of (thousands and repeating) HDFS log lines (on the node) about deleting blocks.

Are these lines unexpected (already have broken RAID controller)?

They keep being scheduled and deleted.

2018-06-14 11:58:53,005 INFO impl.FsDatasetAsyncDiskService (FsDatasetAsyncDiskService.java:run(308)) - Deleted BP-1789482724-9.1.10.22-1491814552298 blk_1155905937_82210162 file /grid/2/hadoop/hdfs/data/current/BP-1789482724-9.1.10.22-1491814552298/current/finalized/subdir229/subdir185/blk_1155905937