Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Cloudera Reporting missing block - HDFS FSCK shows all ok

avatar
Expert Contributor

Hi,

I current have Cloudera and namenode web UI reporting 2 missing blocks.  However on the name node webUI it does not list the block names, just appears empty as per the below:

nnwebuimissingblocks.PNG

However if I run "hdfs fsck /" it does not report any under replicated blocks or corrupt blocks.Current running Cloudera 5.11

hdfsfsck.PNG

Any ideas?

Thanks,
Chris


2 REPLIES 2

avatar
Contributor

Hi, 

 

You have any snapshot enabled? I think that with "hdfs fsck /" you are not checking snapshots.

 

Remove the snapshot with missing blocks and the error will be disappear.

 

From cloudera you can check in Top Menu > Backup > Snapshot Policies

 

The BDR replica can use snapshots automatically, so you need the check from command line:

 

  • Check Snapshotable directories:
$ hdfs lsSnapshottableDir
drwxrwx--- 0 user1 group1 0 2017-08-22 04:00 0 655432 /dir1
  • Remove the snapshot:
$ hdfs dfs -ls /dir1/.snapshot
drwxrwx--- 0 user1 group1 0 2017-08-22 04:00 0 655432 /dir1/.snapshot/<snapshot_id>/dir1

$ hdfs dfs -deleteSnapshot /dir1 <snapshot_id>

 

Check HDFS Snapshots for more information.

 

Regards,

Marc Casajus.

avatar
Expert Contributor

Hi

Thanks for the response, I am not running enterpise or BDR, also I have no snapshots.

I'm really confused as cloudera now reporting 600 corrupt clocks, as does hdfs dfsadmin -report.  However is I run hdfs fsck /, it shows no corrupt block?!?!

Any ideas what would cause the difference between "HDFS DFSADMIN - REPORT" and "hdfs fsck /"