Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

corrupted block issue..i have 100+ corrupted blocks

Solved Go to solution

corrupted block issue..i have 100+ corrupted blocks

Rising Star

I have 4 nodes cluster with NN-HA ..when i restart two datanodes i have 100+ corrupted blocks(shown in ambari>hdfs>servieces ) and i have replication factor 3 so i want to deleted these corrupted blocks..before i deleted this i want to check which are files are corrupted so i run below command and got below ouput ,here did not get any corrupted blocks or files? please help me ..i have this issue since long ..

P1-230-8:/root> hdfs fsck / | egrep -v '^\.+

Connecting to namenode via http://stlpr8711:50070 FSCK started by hdfs (auth:SIMPLE) from /39.6.64.8 for path / at Fri Aug 05 06:27:52 CDT 2016

...........................Status: HEALTHY T

otal size: 39189208876540 B (Total open files size: 83162 B)

Total dirs: 710383 Total files: 2184527

Total symlinks: 0 (Files currently being written: 7) Total blocks (validated): 2309173 (avg. block size 16971101 B) (Total open file blocks (not validated): 7)

Corrupt blocks: 0

Number of data-nodes: 4

Number of racks: 1

FSCK ended at Fri Aug 05 06:28:40 CDT 2016 in 47634 milliseconds The filesystem under path '/' is HEALTHY

1 ACCEPTED SOLUTION

Accepted Solutions

Re: corrupted block issue..i have 100+ corrupted blocks

Hi @sankar rao

This seems to be an known issue with Ambari, please refer below bug for more info and see if version matches.

https://issues.apache.org/jira/browse/AMBARI-12931

7 REPLIES 7

Re: corrupted block issue..i have 100+ corrupted blocks

Contributor
@sankar rao

In the above hdfs fsck command output I see corrupted blocks 0. Generally after you restart the nodes it takes some time for namenode to figure out with blocks location and health state.

Re: corrupted block issue..i have 100+ corrupted blocks

Rising Star

@Vinod Bonthu

Thanks Vinod..but restart happens 20 days back and still in ambari shows that critical warning and 109 corrupted blocks.....

Re: corrupted block issue..i have 100+ corrupted blocks

Contributor

@sankar rao Restarting the Ambari server as work around.

Re: corrupted block issue..i have 100+ corrupted blocks

Hi @sankar rao

This seems to be an known issue with Ambari, please refer below bug for more info and see if version matches.

https://issues.apache.org/jira/browse/AMBARI-12931

Re: corrupted block issue..i have 100+ corrupted blocks

Rising Star

@Jitendra Yadav

Thank you for quick response jitendra..i could not see any version matches..i am using hdp 2.1 with ambari 1.6 can you suggest any other alternative solution for this issue...?

Re: corrupted block issue..i have 100+ corrupted blocks

I would highly suggest you to upgrade your ambari server with latest release. As a workaround restart the Ambari (server and agents) as well as Ambari Metrics and see if that resolve the issue.

Re: corrupted block issue..i have 100+ corrupted blocks

Rising Star

@Jitendra Yadav

Thanks you .. i will let you the result after restart ambari server..