Failure: Upgrade agents in CM v6.3.3

[#1]

I'm testing the CDH upgrade from v5.16.1 to v6.3.3.

Cloudera manager server v6.3.3 upgrade was done successfully.

But I have a trouble in Upgrading the Cloudera Manager Agents section.

https://docs.cloudera.com/cdp-private-cloud/latest/upgrade-cdh/topics/ug_cm_upgrade_agent.html

I can see The Upgrade Cloudera Manager Agent Packages page.

But the Install agents fails due to Failed to copy installation files.

I tried both repository location of Cloudera repository and custom repository but the both was failed.

I think inter hosts ssh communications are no problem because I tested ssh login b/w them and I confirmed the /tmp/scm_prepare_node.XXXXXXXX was made on target agent host.

I suspects key handling is the cause but I'm not sure how to confirm it.

Here's the cloudera-scm-server.log. (modified URL because it's raw username and password)

2020-10-06 14:54:10,500 INFO NodeConfiguratorThread-12-0:com.cloudera.server.cmf.node.NodeConfigurator: Executing mktemp -d /tmp/scm_prepare_node.XXXXXXXX on bsuhdpsv01

2020-10-06 14:54:10,528 INFO NodeConfiguratorThread-12-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: bsuhdpsv01: Transitioning from MAKE_TEMP_DIR (PT0.045S) to COPY_FILES

2020-10-06 14:54:10,736 INFO NodeConfiguratorThread-12-0:com.cloudera.server.cmf.node.NodeConfigurator: Using key bundle from URL: https://****usernameXXX:****[email protected]/p/cm6/6.3.3/allkeys.asc

2020-10-06 14:54:10,745 INFO NodeConfiguratorThread-12-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: bsuhdpsv01: Setting COPY_FILES as failed and done state

2020-10-06 14:54:10,745 INFO NodeConfiguratorThread-12-0:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION

2020-10-06 14:54:15,244 INFO scm-web-2446:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/add-hosts-wizard/installprogressdata.json

2020-10-06 14:54:15,246 INFO scm-web-2446:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/add-hosts-wizard/installprogressdata.json, Status:200

2020-10-06 14:54:15,427 ERROR CommandPusher:com.cloudera.cmf.command.flow.WorkOutputs: CMD id: 1634 Failed to complete installation on host bsuhdpsv01.

2020-10-06 14:54:15,427 ERROR CommandPusher:com.cloudera.cmf.model.DbCommand: Command 1634(GlobalHostInstall) has completed. finalstate:FINISHED, success:false, msg:Failed to complete installation.

Best regards,

Tak567

[#2]

@Tak567 you can manually install/upgrade the agent packages by something like below commands.

# yum install https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/cloudera-manager-agent-6.3.0-1281944.el7.x86_64.rpm
# yum install https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/cloudera-manager-daemons-6.3.0-1281944.el7.x86_64.rpm

Make sure to replace your CM version and user/password accordingly.  

[#3 · Accepted solution]

@GangWar 

Thank you for your reply.

I confirmed I can manually upgrade the agent in each host , which is also written in installation manual as option 2. Upgrade the Agents using the Command Line.

Just a little bit I have a curiosity of the reason not working with CM distributed method, but anyway I'm going forward steps.

Best regards,

Tak568