corrupted block issue..i have 100+ corrupted blocks

[#1]

I have 4 nodes cluster with NN-HA ..when i restart two datanodes i have 100+ corrupted blocks(shown in ambari>hdfs>servieces ) and i have replication factor 3 so i want to deleted these corrupted blocks..before i deleted this i want to check which are files are corrupted so i run below command and got below ouput ,here did not get any corrupted blocks or files? please help me ..i have this issue since long ..

P1-230-8:/root> hdfs fsck / | egrep -v '^\.+

Connecting to namenode via http://stlpr8711:50070 FSCK started by hdfs (auth:SIMPLE) from /39.6.64.8 for path / at Fri Aug 05 06:27:52 CDT 2016

...........................Status: HEALTHY T

otal size: 39189208876540 B (Total open files size: 83162 B)

Total dirs: 710383 Total files: 2184527

Total symlinks: 0 (Files currently being written: 7) Total blocks (validated): 2309173 (avg. block size 16971101 B) (Total open file blocks (not validated): 7)

Corrupt blocks: 0

Number of data-nodes: 4

Number of racks: 1

FSCK ended at Fri Aug 05 06:28:40 CDT 2016 in 47634 milliseconds The filesystem under path '/' is HEALTHY

[#2]

@sankar rao

In the above hdfs fsck command output I see corrupted blocks 0. Generally after you restart the nodes it takes some time for namenode to figure out with blocks location and health state.

[#3]

@Vinod Bonthu

Thanks Vinod..but restart happens 20 days back and still in ambari shows that critical warning and 109 corrupted blocks.....

[#4]

@sankar rao Restarting the Ambari server as work around.

[#5 · Accepted solution]

Hi @sankar rao

This seems to be an known issue with Ambari, please refer below bug for more info and see if version matches.

https://issues.apache.org/jira/browse/AMBARI-12931

[#6]

@Jitendra Yadav

Thank you for quick response jitendra..i could not see any version matches..i am using hdp 2.1 with ambari 1.6 can you suggest any other alternative solution for this issue...?

[#7]

I would highly suggest you to upgrade your ambari server with latest release. As a workaround restart the Ambari (server and agents) as well as Ambari Metrics and see if that resolve the issue.

[#8]

@Jitendra Yadav

Thanks you .. i will let you the result after restart ambari server..