question Re: Need suggestion to stop or Decommission Datanode? in Support Questions

Need suggestion to stop or Decommission Datanode?

madhura2688 — Wed, 24 Apr 2019 22:55:47 GMT

Hi , We are using 12 Datanodes in our cluster. And we have 3 replication factor.

Currently we are doing cost optimization and our total cluster is 35% used. We want to remove one datanode from our cluster, but we want to see first how our cluster performs on 11 datanodes. if it doesn't perform well, we need to restart that datanode again.

To experiment this- Do I need to Stop the datanode or Decomission datanode?

Please need suggestion.

Re: Need suggestion to stop or Decommission Datanode?

Shelton — Sat, 17 Aug 2019 22:47:43 GMT

@Madhura Mhatre

It's well documented in by hortonworks Once you launch the decommissioning the blocks on that node will be distributed to the other node remaining nodes

If the replication factor is higher than the number of existing data nodes after the removal, the removal process is not going to succeed!

Re: Need suggestion to stop or Decommission Datanode?

madhura2688 — Thu, 25 Apr 2019 09:48:22 GMT

@Geoffrey Shelton Okot Thank you for your response. Instead of Decommissioning the node Can I stop it?

What is the difference? Whats the best way to stop it or Decommission?

Re: Need suggestion to stop or Decommission Datanode?

Shelton — Thu, 25 Apr 2019 11:38:22 GMT

@Madhura Mhatre

Surely you could just t^do that but what happens to the replicas stored on that particular data node? Somehow your cluster has to reconstruct those replicas if you had a replicator factor of more than 1.

Here I was talking about planned maintenance! Just switching it off will also force your cluster to do the same thing in the background with alerts and ONLY when the replicas have been reconstructed will those alerts go away.
There is a performance cost for both decommissioning and just unplugging the data node.

Re: Need suggestion to stop or Decommission Datanode?

madhura2688 — Fri, 26 Apr 2019 08:26:20 GMT

@Geoffrey Shelton Okot Thank you. I think Decommissioning is a best way to do this. Do I need to turn on maintainance mode while I do decommissioning datanode? Also Do I need to stop the services like - NodeManager, Ambari metrics for that datanode after decommissioning?