08-03-2017 01:30 AM
I current have Cloudera and namenode web UI reporting 2 missing blocks. However on the name node webUI it does not list the block names, just appears empty as per the below:
However if I run "hdfs fsck /" it does not report any under replicated blocks or corrupt blocks.Current running Cloudera 5.11
08-22-2017 03:45 AM
You have any snapshot enabled? I think that with "hdfs fsck /" you are not checking snapshots.
Remove the snapshot with missing blocks and the error will be disappear.
From cloudera you can check in Top Menu > Backup > Snapshot Policies
The BDR replica can use snapshots automatically, so you need the check from command line:
$ hdfs lsSnapshottableDir
drwxrwx--- 0 user1 group1 0 2017-08-22 04:00 0 655432 /dir1
$ hdfs dfs -ls /dir1/.snapshot
drwxrwx--- 0 user1 group1 0 2017-08-22 04:00 0 655432 /dir1/.snapshot/<snapshot_id>/dir1
$ hdfs dfs -deleteSnapshot /dir1 <snapshot_id>
Check HDFS Snapshots for more information.
08-22-2017 05:50 AM
Thanks for the response, I am not running enterpise or BDR, also I have no snapshots.
I'm really confused as cloudera now reporting 600 corrupt clocks, as does hdfs dfsadmin -report. However is I run hdfs fsck /, it shows no corrupt block?!?!
Any ideas what would cause the difference between "HDFS DFSADMIN - REPORT" and "hdfs fsck /"