Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Clusters cannot be created with a failed/terminated Cloudera Manager.

avatar
Explorer

 Clusters cannot be created with a failed/terminated Cloudera Manager.

 

 

Logs:-

 

[2017-07-12 07:26:49.967 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:26:59.968 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.21.84, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ...

[2017-07-12 07:26:59.968 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection.

[2017-07-12 07:26:59.988 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:00.052 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:00.115 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:00.178 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:01.221 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2)

[2017-07-12 07:27:01.240 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:01.303 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:01.366 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:01.433 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:02.476 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3)

[2017-07-12 07:27:02.497 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:02.559 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:02.623 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:02.685 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:38.243 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:28:05.415 +0000] INFO  [task-thread-5] - - - - c.c.l.m.r.DeploymentsReporter: Enqueueing all deployments for usage reporting

[2017-07-12 07:28:05.418 +0000] INFO  [task-thread-5] - - - - c.c.l.m.r.DeploymentsReporter: Enqueueing 0 deployments for usage reporting

[2017-07-12 07:28:08.243 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.21.84, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ...

[2017-07-12 07:28:08.243 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection.

[2017-07-12 07:28:08.263 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:28:08.326 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:28:08.390 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:28:08.456 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:28:09.499 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2)

[2017-07-12 07:28:09.518 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:28:09.627 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:28:09.688 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:28:09.751 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:28:10.794 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3)

[2017-07-12 07:28:10.813 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:28:10.876 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:28:10.937 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:28:11.000 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:28:11.043 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:28:11.043 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d suspended due to failure working on com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:28:11.044 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d' failed

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5

        at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:0

 

[2017-07-12 07:28:11.044 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d': RUNNING -> SUSPENDED

[2017-07-12 07:28:11.200 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Interrupting runner thread for pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d'

[2017-07-12 07:28:11.200 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d' has already completed

[2017-07-12 07:28:11.207 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d': SUSPENDED -> CANCELED

[2017-07-12 07:28:11.522 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31)

        at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30)

        at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

[2017-07-12 07:28:11.523 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error ErrorInfo{code=ErrorCode{name=JOB_EXECUTION_FAIL, type=SERVICE, retryable=true, keys=[failedJobSize, suspendedJobSize, totalJobSize]}, properties={failedJobSize=0, suspendedJobSize=1, totalJobSize=1}} in job com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines

com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31)

        at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30)

        at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

[2017-07-12 07:28:11.523 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104' failed

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d

        at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:1

 

[2017-07-12 07:28:11.531 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104': RUNNING -> ERROR

[2017-07-12 07:28:11.551 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.d.DeploymentRepositoryService: Deployment 'Alok-test': BOOTSTRAPPING -> BOOTSTRAP_FAILED

[2017-07-12 07:28:14.056 +0000] INFO  [qtp802255644-15] GET /api/v8/environments/Alok-test/deployments/Alok-test/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL

[2017-07-12 07:28:14.056 +0000] INFO  [qtp802255644-15] GET /api/v8/environments/Alok-test/deployments/Alok-test/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE

2 ACCEPTED SOLUTIONS

avatar
Expert Contributor

It looks as though you're specifying the incorrect credentials for the instance that you're using. Often the intended initial username can differ with the AMI, e.g. many CentOS images use "centos" as the username, RHEL often use "ec2-user", etc. Are you using EC2 for this deployment? If so, what AMI are you using and on what region?

View solution in original post

avatar
Explorer

CDH installation stucksCDH installation stucks

# cat cloudera-scm-server.log | grep cdh

 

2017-07-27 22:27:59,881 INFO 1952372338@scm-web-26:com.cloudera.parcel.components.ParcelManagerImpl: Downloading parcels of CDH:5.12.0-1.cdh5.12.0.p0.29 for distros [RHEL7]
2017-07-27 22:27:59,901 INFO 1952372338@scm-web-26:com.cloudera.parcel.components.ParcelDownloaderImpl: Preparing to download: http://archive.cloudera.com/cdh5/parcels/5.12/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:26,159 INFO New I/O  worker #5:com.cloudera.parcel.components.ParcelDownloaderImpl: Completed download of: http://archive.cloudera.com/cdh5/parcels/5.12/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:36,354 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Discovered parcel on CM server: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:36,355 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Created torrent file: /opt/cloudera/parcel-repo/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:36,358 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Creating single-file torrent for CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel...
2017-07-27 22:28:36,359 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashing data from CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel with 2 threads (3248 pieces)...
2017-07-27 22:28:45,278 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   Torrent name: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:50,292 INFO 56131067@scm-web-36:com.cloudera.parcel.components.ParcelManagerImpl: Distributing parcel CDH:5.12.0-1.cdh5.12.0.p0.29 on cluster Cluster
2017-07-27 22:28:50,618 WARN 831920394@scm-web-28:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:50,665 INFO 831920394@scm-web-28:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.24.106 over PT0.047S.  Rate: 1.321 MB/s
2017-07-27 22:28:50,635 WARN 2073794887@scm-web-37:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:50,668 WARN 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:50,681 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.27.115 over PT0.013S.  Rate: 4.778 MB/s
2017-07-27 22:28:50,706 INFO 2073794887@scm-web-37:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.23.5 over PT0.039S.  Rate: 1.593 MB/s
2017-07-27 22:28:50,884 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Parcel download request: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel from : 172.31.24.106
2017-07-27 22:28:50,908 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Served parcel piece CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel (1048576 bytes) to 172.31.24.106 over PT0.012S.  Rate: 83.333 MB/s
2017-07-27 22:32:25,227 INFO 430900238@scm-web-40:com.cloudera.parcel.components.ParcelManagerImpl: Activating parcel CDH:5.12.0-1.cdh5.12.0.p0.29 on cluster Cluster
2017-07-27 22:32:25,261 INFO 430900238@scm-web-40:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command ProcessStalenessCheckCommand BasicCmdArgs{args=[First reason why: com.cloudera.cmf.model.DbCluster.cdhVersionForDb (#1) has changed]}.

 

Also, i have checked full logs without grep and found no error.

 

Please suggest.

 

View solution in original post

18 REPLIES 18

avatar
Expert Contributor

If you are using EBS volumes for your CM/CDH instances, please read the "Pausing a Cluster in AWS" section of the Cloudera Director User Guide. If you are using ephemeral drives, you cannot stop and restart instances, because the data on the ephemeral drives is lost when you restart the instances.

 

avatar
Explorer

Hi Team,

 

If we want to change the password for Cloudera Director, what will be procedure to do it on command line

 

Regards

Alok

avatar
Contributor

 You can use API call:

PUT /api/v2/users/{username}/password

{

"oldPassword" : "foo",

"newPassword": "bar"

}

https://www.cloudera.com/documentation/director/latest/topics/director_server_user_management.html#c...

avatar
Explorer

Hi Team,

 

I am facing issue while CDH installation process on Cloudera director.

 

1) On cloudera director getting error "Bootstrap Failed" while installtion of CDH

2) When i am trying to install CDH thru parcel on Cloudera Manager GUI, there it stucks at Distributing and not going forward.

3) I check and tried with same version for Cloudera Manager and CDH, but still CDH installation do go further.

 

Logs :-

 

[2017-07-27 08:42:29.156 +0000] INFO  [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}
[2017-07-27 08:42:31.181 +0000] INFO  [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}
[2017-07-27 08:42:33.203 +0000] INFO  [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}
[2017-07-27 08:42:35.224 +0000] INFO  [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}
[2017-07-27 08:42:37.246 +0000] INFO  [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}
[2017-07-27 08:42:39.268 +0000] INFO  [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}
[2017-07-27 08:42:41.290 +0000] INFO  [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}
[2017-07-27 08:42:43.314 +0000] INFO  [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}
[2017-07-27 08:42:45.337 +0000] INFO  [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}
[2017-07-27 08:42:47.360 +0000] INFO  [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}

avatar
Contributor

You would like to check cm logs, /var/log/cloudera-manager-server, to see whether there is any trace there.

avatar
Explorer

CDH installation stucksCDH installation stucks

# cat cloudera-scm-server.log | grep cdh

 

2017-07-27 22:27:59,881 INFO 1952372338@scm-web-26:com.cloudera.parcel.components.ParcelManagerImpl: Downloading parcels of CDH:5.12.0-1.cdh5.12.0.p0.29 for distros [RHEL7]
2017-07-27 22:27:59,901 INFO 1952372338@scm-web-26:com.cloudera.parcel.components.ParcelDownloaderImpl: Preparing to download: http://archive.cloudera.com/cdh5/parcels/5.12/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:26,159 INFO New I/O  worker #5:com.cloudera.parcel.components.ParcelDownloaderImpl: Completed download of: http://archive.cloudera.com/cdh5/parcels/5.12/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:36,354 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Discovered parcel on CM server: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:36,355 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Created torrent file: /opt/cloudera/parcel-repo/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:36,358 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Creating single-file torrent for CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel...
2017-07-27 22:28:36,359 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashing data from CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel with 2 threads (3248 pieces)...
2017-07-27 22:28:45,278 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   Torrent name: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:50,292 INFO 56131067@scm-web-36:com.cloudera.parcel.components.ParcelManagerImpl: Distributing parcel CDH:5.12.0-1.cdh5.12.0.p0.29 on cluster Cluster
2017-07-27 22:28:50,618 WARN 831920394@scm-web-28:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:50,665 INFO 831920394@scm-web-28:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.24.106 over PT0.047S.  Rate: 1.321 MB/s
2017-07-27 22:28:50,635 WARN 2073794887@scm-web-37:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:50,668 WARN 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:50,681 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.27.115 over PT0.013S.  Rate: 4.778 MB/s
2017-07-27 22:28:50,706 INFO 2073794887@scm-web-37:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.23.5 over PT0.039S.  Rate: 1.593 MB/s
2017-07-27 22:28:50,884 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Parcel download request: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel from : 172.31.24.106
2017-07-27 22:28:50,908 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Served parcel piece CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel (1048576 bytes) to 172.31.24.106 over PT0.012S.  Rate: 83.333 MB/s
2017-07-27 22:32:25,227 INFO 430900238@scm-web-40:com.cloudera.parcel.components.ParcelManagerImpl: Activating parcel CDH:5.12.0-1.cdh5.12.0.p0.29 on cluster Cluster
2017-07-27 22:32:25,261 INFO 430900238@scm-web-40:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command ProcessStalenessCheckCommand BasicCmdArgs{args=[First reason why: com.cloudera.cmf.model.DbCluster.cdhVersionForDb (#1) has changed]}.

 

Also, i have checked full logs without grep and found no error.

 

Please suggest.

 

avatar
Explorer

Hi All,

 

I am able to completed Cloudera Manager setup (Ver: 5.12) and able to complete CDH (5.12)  installation.

 

But i am unable to download Accumulo and Presto parcel on cloudera manager. Please provide Accumulo and Presto parcel path to download and install on CM.

 

 

 

avatar
New Contributor

Hi Alok,

 

How were you able to resolve SSH key authentication failure. Am facing the same issue.

 

Regards

SMurty

avatar
New Contributor

Hi Alok,

 

Can you explain how you resolved the SSH key error. Am facing the same issue.

 

Thanks!