question Re: How to fix corrupt blocks in Support Questions

How to fix corrupt blocks

gxnan5 — Mon, 30 May 2016 11:54:06 GMT

I make a cluster used ambari included 1 namenode and 4 datanode on vmware. A datanode was broken ,so I deleted this one and created a new datanode .But cluster created CORRUPT blocks .I try to "hdfs fsck -delete / " to fix it, but lost lots of data. how to fix corrupt blocks ?

Re: How to fix corrupt blocks

ssubhas — Mon, 30 May 2016 11:56:01 GMT

@Mon key

Please refer to below link that explains how to find corrupted blocks and fix them.

http://stackoverflow.com/questions/19205057/how-to-fix-corrupt-hadoop-hdfs

Hope this helps.

Thanks and Regards,

Sindhu

Re: How to fix corrupt blocks

nyadav — Mon, 30 May 2016 12:03:07 GMT

What is the replication factor defined, as 1 datanode down should not cause the corruption. Were other datanodes also down at the same time?

Re: How to fix corrupt blocks

sshimpi — Mon, 30 May 2016 12:15:49 GMT

@Mon key

The best way is to find which are corrupted blocks using below command -

 hdfs fsck /path/to/corrupt/file -locations -blocks -files

And then try to manually remove this using "hadoop rm -r </path>" to avoid dataloss.

But still fsck does not remove good copied of data blocks.