question Re: Clusters cannot be created with a failed/terminated Cloudera Manager. in Archives of Support Questions (Read Only)

Clusters cannot be created with a failed/terminated Cloudera Manager.

Argyle — Fri, 16 Sep 2022 15:45:31 GMT

Clusters cannot be created with a failed/terminated Cloudera Manager.

Logs:-

[2017-07-12 07:26:49.967 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

at com.google.common.base.Throwables.propagate(Throwables.java:160)

at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

at com.codahale.metrics.Timer.time(Timer.java:101)

at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

at com.github.rholder.retry.Retryer.call(Retryer.java:160)

at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

at com.github.rholder.retry.Retryer.call(Retryer.java:160)

at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

... 34 common frames omitted

[2017-07-12 07:26:59.968 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.21.84, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ...

[2017-07-12 07:26:59.988 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:00.052 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:00.115 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:00.178 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:01.221 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2)

[2017-07-12 07:27:01.240 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:01.303 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:01.366 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:01.433 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:02.476 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3)

[2017-07-12 07:27:02.497 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:02.559 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:02.623 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:02.685 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:38.243 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed