After a upgrade using Cloudera Manager and pracels I get these events in the active NameNode log.
WARN BlockStateChange: BLOCK* processReport: Report from the DataNode (<datanodeUuid>) is unsorted. This will cause overhead on the NameNode which needs to sort the Full BR. Please update the DataNode to the same version of Hadoop HDFS as the NameNode (2.6.0-cdh5.7.1).
When checking the version of HDFS on the datanode in the event I can see that it reports the correct version.
# hdfs version
Subversion http://github.com/cloudera/hadoop -r ae44a8970a3f0da58d82e0fc65275fff8deabffd
Compiled by jenkins on 2016-06-01T23:26Z
Compiled with protoc 2.5.0
From source with checksum 298b68dc3b308983f04cb37e8416f13
This command was run using /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/jars/hadoop-common-2.6.0-cdh5.7.1.jar
Has anybody seen this warning before?
You can reproduce the error with "hdfs dfsadmin -triggerBlockReport".
We upgraded from CDH5.5.1 -> CDH5.7.1 using parcels.
These logs repeat over the day, upgraded to "5.7.1" 4 days ago and the Warning still show when the any of the nodes do a blockreport.
Didn't do a rolling upgrade, followed the steps in this article.
Looking at the NameNode Web UI / live nodes all of them report version "2.6.0-cdh5.7.1".
I'm running NameNode HA.
Another thing, looking at the "VERSION" file in the DatanNode directories they all have "layoutVersion=-56".
And the NameNode/JournalNode directory "VERSION" file report layoutVersion=-60.
Is this normal, thought they should report the same version?
Thank you for your support on this issue!
We will proceed to upgrade our production cluster to 5.7.1 now that we know the details for this log entry.