Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

Clusters cannot be created with a failed/terminated Cloudera Manager.

Labels:
avatar
author-rank Argyle
Explorer

Created on ‎07-12-2017 01:21 AM - edited ‎09-16-2022 08:45 AM

 Clusters cannot be created with a failed/terminated Cloudera Manager.

 

 

Logs:-

 

[2017-07-12 07:26:49.967 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:26:59.968 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.21.84, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ...

[2017-07-12 07:26:59.968 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection.

[2017-07-12 07:26:59.988 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:00.052 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:00.115 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:00.178 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:01.221 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2)

[2017-07-12 07:27:01.240 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:01.303 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:01.366 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:01.433 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:02.476 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3)

[2017-07-12 07:27:02.497 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:02.559 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:02.623 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:02.685 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:38.243 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:28:05.415 +0000] INFO  [task-thread-5] - - - - c.c.l.m.r.DeploymentsReporter: Enqueueing all deployments for usage reporting

[2017-07-12 07:28:05.418 +0000] INFO  [task-thread-5] - - - - c.c.l.m.r.DeploymentsReporter: Enqueueing 0 deployments for usage reporting

[2017-07-12 07:28:08.243 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.21.84, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ...

[2017-07-12 07:28:08.243 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection.

[2017-07-12 07:28:08.263 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:28:08.326 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:28:08.390 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:28:08.456 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:28:09.499 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2)

[2017-07-12 07:28:09.518 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:28:09.627 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:28:09.688 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:28:09.751 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:28:10.794 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3)

[2017-07-12 07:28:10.813 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:28:10.876 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:28:10.937 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:28:11.000 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:28:11.043 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:28:11.043 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d suspended due to failure working on com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:28:11.044 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d' failed

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5

        at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:0

 

[2017-07-12 07:28:11.044 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d': RUNNING -> SUSPENDED

[2017-07-12 07:28:11.200 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Interrupting runner thread for pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d'

[2017-07-12 07:28:11.200 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d' has already completed

[2017-07-12 07:28:11.207 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d': SUSPENDED -> CANCELED

[2017-07-12 07:28:11.522 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31)

        at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30)

        at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

[2017-07-12 07:28:11.523 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error ErrorInfo{code=ErrorCode{name=JOB_EXECUTION_FAIL, type=SERVICE, retryable=true, keys=[failedJobSize, suspendedJobSize, totalJobSize]}, properties={failedJobSize=0, suspendedJobSize=1, totalJobSize=1}} in job com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines

com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31)

        at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30)

        at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

[2017-07-12 07:28:11.523 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104' failed

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d

        at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:1

 

[2017-07-12 07:28:11.531 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104': RUNNING -> ERROR

[2017-07-12 07:28:11.551 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.d.DeploymentRepositoryService: Deployment 'Alok-test': BOOTSTRAPPING -> BOOTSTRAP_FAILED

[2017-07-12 07:28:14.056 +0000] INFO  [qtp802255644-15] GET /api/v8/environments/Alok-test/deployments/Alok-test/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL

[2017-07-12 07:28:14.056 +0000] INFO  [qtp802255644-15] GET /api/v8/environments/Alok-test/deployments/Alok-test/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE

9,190 Views
0 Kudos
2 ACCEPTED SOLUTIONS

avatar
author-rank Mike Wilson author-rank
Expert Contributor

Created ‎07-12-2017 06:47 AM

It looks as though you're specifying the incorrect credentials for the instance that you're using. Often the intended initial username can differ with the AMI, e.g. many CentOS images use "centos" as the username, RHEL often use "ec2-user", etc. Are you using EC2 for this deployment? If so, what AMI are you using and on what region?

View solution in original post

9,041 Views

avatar
author-rank Argyle
Explorer

Created ‎07-27-2017 10:43 PM

CDH installation stucksCDH installation stucks

# cat cloudera-scm-server.log | grep cdh

 

2017-07-27 22:27:59,881 INFO 1952372338@scm-web-26:com.cloudera.parcel.components.ParcelManagerImpl: Downloading parcels of CDH:5.12.0-1.cdh5.12.0.p0.29 for distros [RHEL7]
2017-07-27 22:27:59,901 INFO 1952372338@scm-web-26:com.cloudera.parcel.components.ParcelDownloaderImpl: Preparing to download: http://archive.cloudera.com/cdh5/parcels/5.12/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:26,159 INFO New I/O  worker #5:com.cloudera.parcel.components.ParcelDownloaderImpl: Completed download of: http://archive.cloudera.com/cdh5/parcels/5.12/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:36,354 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Discovered parcel on CM server: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:36,355 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Created torrent file: /opt/cloudera/parcel-repo/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:36,358 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Creating single-file torrent for CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel...
2017-07-27 22:28:36,359 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashing data from CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel with 2 threads (3248 pieces)...
2017-07-27 22:28:45,278 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   Torrent name: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel
2017-07-27 22:28:50,292 INFO 56131067@scm-web-36:com.cloudera.parcel.components.ParcelManagerImpl: Distributing parcel CDH:5.12.0-1.cdh5.12.0.p0.29 on cluster Cluster
2017-07-27 22:28:50,618 WARN 831920394@scm-web-28:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:50,665 INFO 831920394@scm-web-28:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.24.106 over PT0.047S.  Rate: 1.321 MB/s
2017-07-27 22:28:50,635 WARN 2073794887@scm-web-37:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:50,668 WARN 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent
2017-07-27 22:28:50,681 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.27.115 over PT0.013S.  Rate: 4.778 MB/s
2017-07-27 22:28:50,706 INFO 2073794887@scm-web-37:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.23.5 over PT0.039S.  Rate: 1.593 MB/s
2017-07-27 22:28:50,884 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Parcel download request: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel from : 172.31.24.106
2017-07-27 22:28:50,908 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Served parcel piece CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel (1048576 bytes) to 172.31.24.106 over PT0.012S.  Rate: 83.333 MB/s
2017-07-27 22:32:25,227 INFO 430900238@scm-web-40:com.cloudera.parcel.components.ParcelManagerImpl: Activating parcel CDH:5.12.0-1.cdh5.12.0.p0.29 on cluster Cluster
2017-07-27 22:32:25,261 INFO 430900238@scm-web-40:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command ProcessStalenessCheckCommand BasicCmdArgs{args=[First reason why: com.cloudera.cmf.model.DbCluster.cdhVersionForDb (#1) has changed]}.

 

Also, i have checked full logs without grep and found no error.

 

Please suggest.

 

View solution in original post

10,506 Views
0 Kudos
18 REPLIES 18

avatar
author-rank Mike Wilson author-rank
Expert Contributor

Created ‎07-12-2017 06:47 AM

It looks as though you're specifying the incorrect credentials for the instance that you're using. Often the intended initial username can differ with the AMI, e.g. many CentOS images use "centos" as the username, RHEL often use "ec2-user", etc. Are you using EC2 for this deployment? If so, what AMI are you using and on what region?

9,042 Views

avatar
author-rank Argyle
Explorer

Created ‎07-12-2017 08:00 AM

Hi Mike,

 

Thanks for your quick response.

 

I am using Centos 6.5 and the region on AWS i choosen is Singapore (ap-southeast-1b).

Yes i am using "ec2-user" as username. However on command line i am using "root" as user login name.

 

Please suggest, may i try using username as "root" and try to install.

 

Regards

Alok

Argyle Team

7,680 Views
0 Kudos

avatar
author-rank Mike Wilson author-rank
Expert Contributor

Created ‎07-12-2017 08:31 AM

Yes, so long as you're able to login with it, you can specify it within Director. We'll allow you to specify any arbitrary username to make sure we cover as many edge cases as possible. 🙂 Let me know if this works for you!
7,675 Views
0 Kudos

avatar
author-rank Argyle
Explorer

Created ‎07-18-2017 02:48 AM

Hi Mike,

 

As per the instruction i tried to setup, but still no luc and getting error "Boot Strap Failed"

 

Logs:-

 

[2017-07-18 06:43:55.580 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)
        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)
        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)
        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)
        ... 34 common frames omitted
[2017-07-18 06:44:08.351 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)
        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)
        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)
        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)
        ... 34 common frames omitted
[2017-07-18 06:44:28.352 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.19.72, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ...
[2017-07-18 06:44:28.352 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection.
[2017-07-18 06:44:28.372 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.19.72 was automatically accepted
[2017-07-18 06:44:28.436 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-19-72.ap-southeast-1.compute.internal was automatically accepted
[2017-07-18 06:44:28.499 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 54.169.68.43 was automatically accepted
[2017-07-18 06:44:28.562 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-54-169-68-43.ap-southeast-1.compute.amazonaws.com was automatically accepted
[2017-07-18 06:44:29.605 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2)
[2017-07-18 06:44:29.625 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.19.72 was automatically accepted
[2017-07-18 06:44:29.688 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-19-72.ap-southeast-1.compute.internal was automatically accepted
[2017-07-18 06:44:29.751 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 54.169.68.43 was automatically accepted
[2017-07-18 06:44:29.813 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-54-169-68-43.ap-southeast-1.compute.amazonaws.com was automatically accepted
[2017-07-18 06:44:30.856 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3)
[2017-07-18 06:44:30.876 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.19.72 was automatically accepted
[2017-07-18 06:44:30.938 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-19-72.ap-southeast-1.compute.internal was automatically accepted
[2017-07-18 06:44:31.001 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 54.169.68.43 was automatically accepted
[2017-07-18 06:44:31.065 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-54-169-68-43.ap-southeast-1.compute.amazonaws.com was automatically accepted
[2017-07-18 06:44:31.111 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)
        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)
        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)
        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)
        ... 34 common frames omitted
[2017-07-18 06:45:03.892 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)
        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)
        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)
        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)
        ... 34 common frames omitted
[2017-07-18 06:45:03.893 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b suspended due to failure working on com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation
java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)
        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)
        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)
        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)
        ... 34 common frames omitted
[2017-07-18 06:45:03.896 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b' failed
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a
        at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:0

[2017-07-18 06:45:03.896 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b': RUNNING -> SUSPENDED
[2017-07-18 06:45:04.223 +0000] INFO  [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Interrupting runner thread for pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b'
[2017-07-18 06:45:04.223 +0000] INFO  [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b' has already completed
[2017-07-18 06:45:04.240 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b': SUSPENDED -> CANCELED
[2017-07-18 06:45:04.532 +0000] ERROR [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78)
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31)
        at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30)
        at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$be60b702.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
[2017-07-18 06:45:04.535 +0000] ERROR [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error ErrorInfo{code=ErrorCode{name=JOB_EXECUTION_FAIL, type=SERVICE, retryable=true, keys=[failedJobSize, suspendedJobSize, totalJobSize]}, properties={failedJobSize=0, suspendedJobSize=1, totalJobSize=1}} in job com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines
com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78)
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31)
        at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30)
        at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$be60b702.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
[2017-07-18 06:45:04.535 +0000] ERROR [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff' failed
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$be60b702
        at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:1

[2017-07-18 06:45:04.540 +0000] INFO  [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.s.PipelineRepositoryService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff': RUNNING -> ERROR
[2017-07-18 06:45:04.559 +0000] INFO  [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.d.DeploymentRepositoryService: Deployment 'testing': BOOTSTRAPPING -> BOOTSTRAP_FAILED
[2017-07-18 06:45:05.293 +0000] INFO  [qtp802255644-119] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL
[2017-07-18 06:45:05.293 +0000] INFO  [qtp802255644-119] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE
[2017-07-18 06:45:10.297 +0000] INFO  [qtp802255644-14] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL
[2017-07-18 06:45:10.297 +0000] INFO  [qtp802255644-14] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE
[2017-07-18 06:45:15.295 +0000] INFO  [qtp802255644-115] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL
[2017-07-18 06:45:15.295 +0000] INFO  [qtp802255644-115] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE
[2017-07-18 06:45:20.327 +0000] INFO  [qtp802255644-16] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL
[2017-07-18 06:45:20.327 +0000] INFO  [qtp802255644-16] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE
[2017-07-18 06:45:23.724 +0000] INFO  [task-thread-10] - - - - c.c.launchpad.task.RefreshClusters: Refreshing Cluster models
[2017-07-18 06:45:23.725 +0000] INFO  [task-thread-10] - - - - c.c.launchpad.task.RefreshClusters: Finished refreshing all pre-existing Cluster models
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Refreshing pre-existing Deployments
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:Argyle as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:Alok as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:ALok as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:test as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment alok14:alok-alok as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Alok-test:testing as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Finished refreshing all pre-existing Deployment models
[2017-07-18 06:45:25.307 +0000] INFO  [qtp802255644-19] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL
[2017-07-18 06:45:25.308 +0000] INFO  [qtp802255644-19] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE

7,613 Views
0 Kudos

avatar
author-rank Mike Wilson author-rank
Expert Contributor

Created ‎07-18-2017 01:09 PM

It looks as though the user authentication is still the problem. Can you try provisioning an instance of the AMI you're trying to use manually and attempting to connect using the credentials you're providing to Director?

i.e. Launch an instance using the EC2 console and just attempting to SSH into it.
7,599 Views
0 Kudos

avatar
author-rank Argyle
Explorer

Created ‎07-18-2017 09:50 PM

Mike,

 

I already checked the same and able to ssh the instance.

 

Regards

Alok Katiyar

7,585 Views
0 Kudos

avatar
author-rank Mike Wilson author-rank
Expert Contributor

Created ‎07-19-2017 11:20 AM

Okay, hrm. Are you using the web UI, or a conf file to set up this cluster? If you're using a conf file, would you be able to post it on something like pastebin (being sure to redact anything sensitive like AWS secret keys, access keys, and SSH keys) and link it here? Also would you be able to give me the exact AMI ID that's being used?

Are you certain you're using the same private key when trying to set up a cluster that you're using to SSH into raw instances? This error is still consistent with incorrect SSH credentials being supplied -- we just have to figure out exactly what's going on here.
7,557 Views
0 Kudos

avatar
author-rank Argyle
Explorer

Created ‎07-19-2017 09:40 PM

Hello Mike,

 

I am able to resolve the issue and able to deploy Cloudera Manager, I am into more troubleshooting and will catch you soon with more questions..lolz..

 

Hope you dont mind. Thanks for the support.

 

Regards

Alok

7,551 Views
0 Kudos

avatar
author-rank Argyle
Explorer

Created ‎07-20-2017 06:58 AM

Hello Mike,

 

Require small information, when i turn off the instances and start again my cluster is not coming up, moreover my Cloudera Director is working fine but the cluster made under enviorment is done bad and my instances (Master and worker ) are not coming up and i have to Terminate the cluster and need to re-create again.

 

Also, i am unable to add services and getting error for "cdh" i tried to install "cdh" but it stuck to an point.

 

Kindly interven and require your inputs.

 

Regards

Alok

7,538 Views
0 Kudos
Announcements
What's New @ Cloudera
Cloudera DataFlow 2.9 now supports building GenAI pipelines ...
What's New @ Cloudera
Accelerate Data Scientist Productivity with Cloudera Copilot...
What's New @ Cloudera
[RELEASED] Cloudera Streaming Analytics - Kubernetes Operato...
Community Announcements
October 2024 Community Highlights
What's New @ Cloudera
Accelerate Your AI with the Cloudera AI Inference Service wi...
View More Announcements