We have completely same problem as https://issues.apache.org/jira/browse/HDFS-11797 with HDP 2.6.1.
https://issues.apache.org/jira/browse/HDFS-11797?focusedCommentId=16039577&page=com.atlassian.jira.p...
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Inconsistent number of corrupt replicas for blk_123456789_123456 blockMap has 0 but corrupt replicas map has 1
org.apache.hadoop.ipc.Server: IPC Server handler 34 on 8020, call org.apache.hadoop.hdfs.protocol.ClientProtocol.getListing from xxx.xxx.xxx.xxx:xxxxx Call#91 Retry#0 java.lang.ArrayIndexOutOfBoundsException
Actually, our hive client fails for this problem. and hdfs fsck command also fails for this hdfs file with this problem.
I read a series of JIRA tickets.
https://issues.apache.org/jira/browse/HDFS-9958
https://issues.apache.org/jira/browse/HDFS-10788
https://issues.apache.org/jira/browse/HDFS-11797
https://issues.apache.org/jira/browse/HDFS-11445
https://issues.apache.org/jira/browse/HDFS-11755
At the second last comment of HDFS-11755
https://issues.apache.org/jira/browse/HDFS-11755?focusedCommentId=16200946&page=com.atlassian.jira.p...
As discussed in HDFS-11445, a regression caused by HDFS-11445 is fixed by HDFS-11755. I'd like to backport HDFS-11755 into branch-2.7 as a result.
and,
https://issues.apache.org/jira/browse/HDFS-11755?focusedCommentId=16201164&page=com.atlassian.jira.p...
Filed HDFS-12641 to initiate the discussion.
and https://issues.apache.org/jira/browse/HDFS-12641 is not resolved.
I'm not sure, but HDFS-12641 may be only for CDH???
I've also checked, HDFS-11445 is not included in HDP 2.6.1. but, it's included in HDP 2.6.2.
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_release-notes/content/patch_hadoop.html
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_release-notes/content/patch_hadoop.html
So, can someone confirm that our current problem with message "blockMap has 0 but corrupt replicas map has 1" is safely fixed with HDP 2.6.2 with HDFS-11445?
Actually, we plan to upgrade from HDP 2.6.1 to HDP 2.6.2. but I'm worry about that upgrading to HDP 2.6.2 would make new problem such as HDFS-11755 says "a regression caused by HDFS-11445 is fixed by HDFS-11755."...
I've confirmed HDFS-11755 is not included in HDP 2.6.1, HDP 2.6.2.
