question Re: How to remove risk disks from Hadoop cluster ? in Support Questions

How to remove risk disks from Hadoop cluster ?

Hefei — Sun, 18 Aug 2019 12:51:43 GMT

Some disks failed in my HDFS cluster. and nodemanager cannot start in these nodes.

How to fix it ?

Re: How to remove risk disks from Hadoop cluster ?

divakarreddy_a — Mon, 22 Feb 2016 14:26:14 GMT

Is multiple disks on the same node?

if yes, I think you can decommission the node from Ambari->hosts->data node -> you will find decommission from drop down.

Re: How to remove risk disks from Hadoop cluster ?

jstraub — Mon, 22 Feb 2016 14:41:14 GMT

Take a look at this questions, maybe it is helpful => https://community.hortonworks.com/questions/3012/what-are-the-steps-an-operator-should-take-to-repl.html

Re: How to remove risk disks from Hadoop cluster ?

rahulpathak109 — Mon, 22 Feb 2016 14:50:43 GMT

In Namenode UI check and ensure that there are no missing and corrupt blocks. If this is true then you can successfully remove failed disk from DataNode.

Refer this for details

Re: How to remove risk disks from Hadoop cluster ?

Hefei — Mon, 22 Feb 2016 15:11:49 GMT

Thanks for quick reply.

Re: How to remove risk disks from Hadoop cluster ?

Hefei — Mon, 22 Feb 2016 15:11:54 GMT

Thanks for quick reply.

Re: How to remove risk disks from Hadoop cluster ?

bleonhardi — Mon, 22 Feb 2016 17:21:29 GMT

Can you check the setting for

dfs.datanode.failed.volumes.tolerated

in your environment. Default is 0 which is a bit restrictive. Normally 1 or even 2 ( on datanodes with high disc density ) make more operational sense.

Then your datanode will start and you can take care of the discs.

Re: How to remove risk disks from Hadoop cluster ?

aervits — Mon, 22 Feb 2016 20:33:49 GMT

Iif you put this machine in a separate config group and remove referencw to the directories used you can keep the machine up. Removing disk and not replacing will mean your data will be writing to OS filesystem. Also do what Benjamin siggests and increase tolerance.