Having issues updating CM

[#1]

Please note that I am very new to both the Cloudera and Linux worlds.  I have a completely development environment that I am playing with, trying to get my feet wet.  I have been successful in doing a minor CDH upgrade from 5.3.2 to 5.3.8 but I am wanting to understand how to to a more major version upgrade.  The version that I am trying to get to is 5.4.1-1. 

From what I have read, when doing a major version change, the CM has to be updated first.  This is supported in my mind by the fact that CM seems not to recognize the 5.4.1-1 parcel I have placed in the same directory as my earlier upgrade that worked flawlessly.  This was done using the Upgrade Cluster wizard.

I have tried following the directions here but get stuck under the instructions to "Upgrade Cloudera Manager Server (Packages)"  I assume this is my ignorance of Linux shell command but I have no idea that the statements:

[cloudera-manager]
# Packages for Cloudera Manager, Version 5, on RedHat or CentOS 6 x86_64
name=Cloudera Manager
baseurl=http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5/
gpgkey = http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/RPM-GPG-KEY-cloudera
gpgcheck = 1

mean?

It seems to me as though this is going out to look for the CM version and pull it down but I am not certain.

It would be a great help if anyone could please help me out and let me know if I am even headed in the right direction here.

Thank you!

[#2]

To upgrade Clouder Manager is pretty much just like upgrading any software in the linux world.   If you are not familiar with how to do that it could be confusing.

You are correct, the CM version must be equal or higher than the CDH version.  So if you want CDH 5.4.x, you need to have CM5.4.x or higher.

What O/S and version are you running CM on?    Maybe I can give you some quick and instructions.   You say you are playing around, so my directions will not have all the cautions you should be looking at if you have data you are tyring to save.  You will have to decipher the docs for that.

Tina

[#3]

Thank you very much for your reply!  I do not have any data that is important in this environment.  In this cluster I am running RHEL 6.5.

I have also found, perhaps, a little more information:  There is a parcel available to move to 5.12.1-1.  I can distribute and activate this.  However, if I do, HDFS becomes unhappy and will not allow be to finalize.  After looking into the steps of updating HDFS it seems as or more complicated than where I am with updating CM.  The error that I am getting with HDFS is:

2014-03-13 23:38:15,488 FATAL org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode join
java.io.IOException:
File system image contains an old layout version -40.
An upgrade to version -47 is required.
Please restart NameNode with -upgrade option.
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:221)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:787)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:568)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:443)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:491)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:684)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:669)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1254)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1320)
2014-03-13 23:38:15,492 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2014-03-13 23:38:15,493 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at nn1/192.168.2.202
************************************************************/
~

---

This is arbitrary sample data as I have already rolled back and cleared these errors but where this says -40 mine said -59 and required -60.

[#4 · Accepted solution]

Not sure what that hdfs error is, probably becuae you did not upgrade the CM Server. 

Here are some quick directions.   You will need to login to the host running CM Server and the hosts running the CM agents.

1) Stop Cloudera manager services in the Cloudera Manager UI
2) Stop All the cluster services (probably don't need to stop all of them, but it is easier and faster)

3) login to the host running cloudera manger server

4) Stop cloudera manager and the DB, (db only if you are using embedded postgres)

service cloudera-scm-server stop
service cloudera-scm-server-db stop

5) see if the new version of CM is available to upgrade

yum clean all
yum clean metadata

yum list available | grep -i cloudera-manager-

should show the latest version of CM available for you to upgrade to.

6) to upgrade CM

yum udpate -y  'cloudera-manager-*'

7) Verify the correct versions were installed

rpm -qa 'cloudera-manager-*'

😎 Upgrade the CM agents by logging into each host and follow the same commands.

9) Start all the servers 

service cloudera-scm-server-db start
service cloudera-scm-server start
service cloudera-scm-agent start

10) when they have started, log back into the CM UI and start the Cloudera Managment Services and the CDH Services.

You can google all these commands for better undersanding of what they are and trouble shooting.

Good Luck!

Tina

[#5]

Thank you very much for the reply.  I apologize but I havent had time to get through this until now.  When searching for available updates I am getting the following:

yum list available | grep -i cloudera-manager-
Repo rhel-server-dts2-6-rpms forced skip_if_unavailable=True due to: /etc/rhsm/ca/redhat-uep.pem
Repo rhel-server-dts2-6-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/8878952237103311606-key.pem
Repo rhel-server-dts-6-rpms forced skip_if_unavailable=True due to: /etc/rhsm/ca/redhat-uep.pem
Repo rhel-server-dts-6-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/8878952237103311606-key.pem
Repo rhel-6-server-rpms forced skip_if_unavailable=True due to: /etc/rhsm/ca/redhat-uep.pem
Repo rhel-6-server-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/8878952237103311606-key.pem
Repo rhel-6-server-eus-rpms forced skip_if_unavailable=True due to: /etc/rhsm/ca/redhat-uep.pem
Repo rhel-6-server-eus-rpms forced skip_if_unavailable=True due to: /etc/pki/entitlement/8878952237103311606-key.pem

Not certain why the update would be unavailable across all repos.  I am running RHEL 6.5.  Everything seemed to work normally up until this point.

[#6]

As it turns out these force skips were due to the cloudera-manager.repo file being altered during initial installation of this cluster to make sure it didnt get updated.  After editing this file back to default everything started to work as normal.

[#7]

Yeah!  Great news!