Error while adding new host to cluster

[#1]

Am trying to add new host to cluster, but I keep getting this error message `https://archive.cloudera.com/cm6/6.0.1/allkeys.asc/`, logs show same message without details

[#2]

Hello @Exor , 

The issue seems to be that the agent is not able to connect to archive.cloudera.com. 

Could you check or share what the failure is via the log files present under /tmp/scm_prepare_node.*/*

[#3]

there no log files in that dir, only the below 3 files, also .sh is not executable

Logs on the master server

2021-06-15 08:39:14,006 INFO scm-web-7889:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/add-hosts-wizard/installretry.json
2021-06-15 08:39:14,007 INFO scm-web-7889:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command GlobalHostInstall GlobalHostInstallCommandArgs{sshPort=22, userName=sw, password=REDACTED, passphrase=REDACTED, privateKey=REDACTED, parallelInstallCount=10, cmRepoUrl=null, gpgKeyCustomUrl=null, gpgKeyOverrideBundle=<none>, unlimitedJCE=false, javaInstallStrategy=NONE, agentUserMode=ROOT, cdhVersion=-1, cdhRelease=NONE>, cdhRepoUrl=null, buildCertCommand=, sslCertHostname=null, reqId=6, skipPackageInstall=false, skipCloudConfig=false, hosts=[node5], existingHosts=[]}.
2021-06-15 08:39:14,008 INFO scm-web-7889:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in sequence
2021-06-15 08:39:14,008 INFO scm-web-7889:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Install on 1 hosts.
2021-06-15 08:39:14,008 INFO scm-web-7889:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Install on node5.
2021-06-15 08:39:14,008 INFO scm-web-7889:com.cloudera.server.cmf.node.NodeConfiguratorService: Adding password-based configurator for node5
2021-06-15 08:39:14,008 INFO scm-web-7889:com.cloudera.server.cmf.node.NodeConfiguratorService: Submitted configurator for node5 with id 7
2021-06-15 08:39:14,011 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Transitioning from INIT (PT0.003S) to CONNECT
2021-06-15 08:39:14,012 INFO NodeConfiguratorThread-6-0:net.schmizz.sshj.transport.TransportImpl: Client identity string: SSH-2.0-SSHJ_0_14_0
2021-06-15 08:39:14,014 INFO scm-web-7889:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/add-hosts-wizard/installretry.json, Status:200
2021-06-15 08:39:14,032 INFO NodeConfiguratorThread-6-0:net.schmizz.sshj.transport.TransportImpl: Server identity string: SSH-2.0-OpenSSH_6.4
2021-06-15 08:39:14,056 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Transitioning from CONNECT (PT0.045S) to AUTHENTICATE
2021-06-15 08:39:14,130 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Transitioning from AUTHENTICATE (PT0.074S) to MAKE_TEMP_DIR
2021-06-15 08:39:14,258 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfigurator: Executing mktemp -d /tmp/scm_prepare_node.XXXXXXXX on node5
2021-06-15 08:39:14,301 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Transitioning from MAKE_TEMP_DIR (PT0.171S) to COPY_FILES
2021-06-15 08:39:14,498 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfigurator: Using key bundle from URL: https://archive.cloudera.com/cm6/6.0.1/allkeys.asc
2021-06-15 08:39:14,960 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Setting COPY_FILES as failed and done state
2021-06-15 08:39:14,960 INFO NodeConfiguratorThread-6-0:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION
2021-06-15 08:39:19,024 ERROR CommandPusher:com.cloudera.cmf.command.flow.WorkOutputs: CMD id: 12141 Failed to complete installation on host node5.
2021-06-15 08:39:19,024 ERROR CommandPusher:com.cloudera.cmf.model.DbCommand: Command 12141(GlobalHostInstall) has completed. finalstate:FINISHED, success:false, msg:Failed to complete installation.
2021-06-15 08:39:19,661 INFO scm-web-7049:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/add-hosts-wizard/installprogressdata.json
2021-06-15 08:39:19,662 INFO scm-web-7049:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/add-hosts-wizard/installprogressdata.json, Status:200
2021-06-15 08:39:24,769 INFO avro-servlet-hb-processor-1:com.cloudera.server.common.AgentAvroServlet: (15 skipped) AgentAvroServlet: heartbeat processing stats: average=12ms, min=4ms, max=38ms.

[#4]

Hi @Exor ,

I am guessing that the key part of the log for this issue is this:

2021-06-15 08:39:14,498 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfigurator: Using key bundle from URL: https://archive.cloudera.com/cm6/6.0.1/allkeys.asc
2021-06-15 08:39:14,960 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Setting COPY_FILES as failed and done state

The log doesn't indicate it, but the server archive.cloudera.com is probably returning an HTTP 401 error because authentication is required.

You didn't provide the version of Cloudera Manager you're using, but I would guess that the last time you attempted this operation, you were not challenged for authentication by this particular host at Cloudera and so you're wondering what changed recently.

The answer is probably that your installation of Cloudera Manager isn't set up to supply the authentication credentials. Earlier this year, Cloudera announced new versions of Cloudera Manager 6.x that are required to access Cloudera’s repositories. Please see the announcement here: Transition to private repositories for CDH, HDP and HDF.

The same announcement describes the new patch releases of Cloudera Manager, which are required to access Cloudera’s private repositories, which now contain the new and legacy releases and other assets such as those necessary to add a new host to an existing CDH cluster.

[#5]

Hi,

In case we have set up a local repository, is there a chance to find the file allkeys.asc from somewhere and place it in cloudera-repos/cm6 directory? We have a cloudera cluster installed and running from before February 2021 (unfortunately it was shut down for a few months), is it possible to use the already installed packages/parcels in cloudera-manager machine and host machines to set up properly the local repository?

Thank you in advance,

Alexander