Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Datanode issue after CDH5.2 upgrade

Datanode issue after CDH5.2 upgrade

Explorer
All,
 
 
I tried to upgrade my cluster from 5.0.1 to 5.2 following the documentation http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_mc_upgrade_to_cdh52_...
 
Everything went fine , except the datanode when starting it , i got the exception below.
 
2014-12-15 15:28:53,734 INFO org.apache.hadoop.hdfs.server.common.Storage: Upgrading block pool storage directory /NAS/dfs/dn/current/BP-1056675496-10.15.101.22-1402685851165.
   old LV = -55; old CTime = 0.
   new LV = -56; new CTime = 1418674697177
2014-12-15 15:28:57,228 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to Namenode/10.15.101.22:8022. Exiting.
java.io.IOException: EEXIST: File exists
 
 
To resolve this i manually have edited  the VERSION on dfs directories and changed the ctime field and version fields to read  and , replaced the namenode metadata backup that i have taken before upgrade.
 
Now when i bring up the cluster , i see namenode reports lot of blocks as missing, i tried to find out the blocks that were reported missing and i couldnt find it anywhere, please help to resolve this issue.
 
I cannot afford to wipe out everything on  DN/NN dfs directories  as the data in the cluster is heavily used.
 

Regards

Suresh
3 REPLIES 3

Re: Datanode issue after CDH5.2 upgrade

The upgrade to v5.2 is not the same as in earlier minor version upgrades.
There has been a version upgrade to HDFS metadata. Please undo the manual
change to the Datanodes. Then follow this upgrade guide

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_mc_upgrade_to_cdh52....

Regards,
Gautam Gopalakrishnan

Re: Datanode issue after CDH5.2 upgrade

Explorer

Gautam,

 

I have followed the same documentation and have even  mentioned that in my original note.

 

Regards

Suresh

 

 

Highlighted

Re: Datanode issue after CDH5.2 upgrade

Expert Contributor

Have you tried upgrading it again from cloudera console?  I had similar issues and after removing the cluster node and upgrading, adding it back resolved the issue without deleting any data.

 

It worked pretty smooth.

 

Let me know how it goes.

Don't have an account?
Coming from Hortonworks? Activate your account here