question Re: Error while adding new host to cluster in Support Questions

Error while adding new host to cluster

Exor — Mon, 14 Jun 2021 10:58:07 GMT

Am trying to add new host to cluster, but I keep getting this error message `https://archive.cloudera.com/cm6/6.0.1/allkeys.asc/`, logs show same message without details

Re: Error while adding new host to cluster

tjangid — Mon, 14 Jun 2021 15:08:49 GMT

Hello @Exor ,

The issue seems to be that the agent is not able to connect to archive.cloudera.com.

Could you check or share what the failure is via the log files present under /tmp/scm_prepare_node.*/*

Re: Error while adding new host to cluster

Exor — Tue, 15 Jun 2021 06:42:34 GMT

there no log files in that dir, only the below 3 files, also .sh is not executable

Logs on the master server

2021-06-15 08:39:14,006 INFO scm-web-7889:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/add-hosts-wizard/installretry.json 2021-06-15 08:39:14,007 INFO scm-web-7889:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command GlobalHostInstall GlobalHostInstallCommandArgs{sshPort=22, userName=sw, password=REDACTED, passphrase=REDACTED, privateKey=REDACTED, parallelInstallCount=10, cmRepoUrl=null, gpgKeyCustomUrl=null, gpgKeyOverrideBundle=<none>, unlimitedJCE=false, javaInstallStrategy=NONE, agentUserMode=ROOT, cdhVersion=-1, cdhRelease=NONE>, cdhRepoUrl=null, buildCertCommand=, sslCertHostname=null, reqId=6, skipPackageInstall=false, skipCloudConfig=false, hosts=[node5], existingHosts=[]}. 2021-06-15 08:39:14,008 INFO scm-web-7889:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in sequence 2021-06-15 08:39:14,008 INFO scm-web-7889:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Install on 1 hosts. 2021-06-15 08:39:14,008 INFO scm-web-7889:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Install on node5. 2021-06-15 08:39:14,008 INFO scm-web-7889:com.cloudera.server.cmf.node.NodeConfiguratorService: Adding password-based configurator for node5 2021-06-15 08:39:14,008 INFO scm-web-7889:com.cloudera.server.cmf.node.NodeConfiguratorService: Submitted configurator for node5 with id 7 2021-06-15 08:39:14,011 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Transitioning from INIT (PT0.003S) to CONNECT 2021-06-15 08:39:14,012 INFO NodeConfiguratorThread-6-0:net.schmizz.sshj.transport.TransportImpl: Client identity string: SSH-2.0-SSHJ_0_14_0 2021-06-15 08:39:14,014 INFO scm-web-7889:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/add-hosts-wizard/installretry.json, Status:200 2021-06-15 08:39:14,032 INFO NodeConfiguratorThread-6-0:net.schmizz.sshj.transport.TransportImpl: Server identity string: SSH-2.0-OpenSSH_6.4 2021-06-15 08:39:14,056 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Transitioning from CONNECT (PT0.045S) to AUTHENTICATE 2021-06-15 08:39:14,130 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Transitioning from AUTHENTICATE (PT0.074S) to MAKE_TEMP_DIR 2021-06-15 08:39:14,258 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfigurator: Executing mktemp -d /tmp/scm_prepare_node.XXXXXXXX on node5 2021-06-15 08:39:14,301 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Transitioning from MAKE_TEMP_DIR (PT0.171S) to COPY_FILES 2021-06-15 08:39:14,498 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfigurator: Using key bundle from URL: https://archive.cloudera.com/cm6/6.0.1/allkeys.asc 2021-06-15 08:39:14,960 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Setting COPY_FILES as failed and done state 2021-06-15 08:39:14,960 INFO NodeConfiguratorThread-6-0:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION 2021-06-15 08:39:19,024 ERROR CommandPusher:com.cloudera.cmf.command.flow.WorkOutputs: CMD id: 12141 Failed to complete installation on host node5. 2021-06-15 08:39:19,024 ERROR CommandPusher:com.cloudera.cmf.model.DbCommand: Command 12141(GlobalHostInstall) has completed. finalstate:FINISHED, success:false, msg:Failed to complete installation. 2021-06-15 08:39:19,661 INFO scm-web-7049:com.cloudera.enterprise.JavaMelodyFacade: Entering HTTP Operation: Method:POST, Path:/add-hosts-wizard/installprogressdata.json 2021-06-15 08:39:19,662 INFO scm-web-7049:com.cloudera.enterprise.JavaMelodyFacade: Exiting HTTP Operation: Method:POST, Path:/add-hosts-wizard/installprogressdata.json, Status:200 2021-06-15 08:39:24,769 INFO avro-servlet-hb-processor-1:com.cloudera.server.common.AgentAvroServlet: (15 skipped) AgentAvroServlet: heartbeat processing stats: average=12ms, min=4ms, max=38ms.

Re: Error while adding new host to cluster

ask_bill_brooks — Thu, 17 Jun 2021 15:34:53 GMT

Hi @Exor ,

I am guessing that the key part of the log for this issue is this:

2021-06-15 08:39:14,498 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfigurator: Using key bundle from URL: https://archive.cloudera.com/cm6/6.0.1/allkeys.asc 2021-06-15 08:39:14,960 INFO NodeConfiguratorThread-6-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: node5: Setting COPY_FILES as failed and done state

The log doesn't indicate it, but the server archive.cloudera.com is probably returning an HTTP 401 error because authentication is required.

You didn't provide the version of Cloudera Manager you're using, but I would guess that the last time you attempted this operation, you were not challenged for authentication by this particular host at Cloudera and so you're wondering what changed recently.

The answer is probably that your installation of Cloudera Manager isn't set up to supply the authentication credentials. Earlier this year, Cloudera announced new versions of Cloudera Manager 6.x that are required to access Cloudera’s repositories. Please see the announcement here: Transition to private repositories for CDH, HDP and HDF.

The same announcement describes the new patch releases of Cloudera Manager, which are required to access Cloudera’s private repositories, which now contain the new and legacy releases and other assets such as those necessary to add a new host to an existing CDH cluster.

Re: Error while adding new host to cluster

alex92 — Wed, 18 Aug 2021 12:14:20 GMT

Hi,

In case we have set up a local repository, is there a chance to find the file allkeys.asc from somewhere and place it in cloudera-repos/cm6 directory? We have a cloudera cluster installed and running from before February 2021 (unfortunately it was shut down for a few months), is it possible to use the already installed packages/parcels in cloudera-manager machine and host machines to set up properly the local repository?

Thank you in advance,

Alexander