Reply
Explorer
Posts: 10
Registered: ‎07-12-2017
Accepted Solution

Clusters cannot be created with a failed/terminated Cloudera Manager.

 Clusters cannot be created with a failed/terminated Cloudera Manager.

 

 

Logs:-

 

[2017-07-12 07:26:49.967 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:26:59.968 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.21.84, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ...

[2017-07-12 07:26:59.968 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection.

[2017-07-12 07:26:59.988 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:00.052 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:00.115 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:00.178 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:01.221 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2)

[2017-07-12 07:27:01.240 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:01.303 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:01.366 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:01.433 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:02.476 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3)

[2017-07-12 07:27:02.497 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:27:02.559 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:27:02.623 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:27:02.685 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:27:38.243 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:28:05.415 +0000] INFO  [task-thread-5] - - - - c.c.l.m.r.DeploymentsReporter: Enqueueing all deployments for usage reporting

[2017-07-12 07:28:05.418 +0000] INFO  [task-thread-5] - - - - c.c.l.m.r.DeploymentsReporter: Enqueueing 0 deployments for usage reporting

[2017-07-12 07:28:08.243 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.21.84, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ...

[2017-07-12 07:28:08.243 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection.

[2017-07-12 07:28:08.263 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:28:08.326 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:28:08.390 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:28:08.456 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:28:09.499 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2)

[2017-07-12 07:28:09.518 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:28:09.627 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:28:09.688 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:28:09.751 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:28:10.794 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3)

[2017-07-12 07:28:10.813 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted

[2017-07-12 07:28:10.876 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted

[2017-07-12 07:28:10.937 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted

[2017-07-12 07:28:11.000 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted

[2017-07-12 07:28:11.043 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:28:11.043 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d suspended due to failure working on com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation

java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at com.google.common.base.Throwables.propagate(Throwables.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)

        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)

        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods

        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)

        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)

        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)

        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)

        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)

        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)

        ... 34 common frames omitted

[2017-07-12 07:28:11.044 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d' failed

        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5

        at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:0

 

[2017-07-12 07:28:11.044 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d': RUNNING -> SUSPENDED

[2017-07-12 07:28:11.200 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Interrupting runner thread for pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d'

[2017-07-12 07:28:11.200 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d' has already completed

[2017-07-12 07:28:11.207 +0000] INFO  [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d': SUSPENDED -> CANCELED

[2017-07-12 07:28:11.522 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31)

        at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30)

        at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

[2017-07-12 07:28:11.523 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error ErrorInfo{code=ErrorCode{name=JOB_EXECUTION_FAIL, type=SERVICE, retryable=true, keys=[failedJobSize, suspendedJobSize, totalJobSize]}, properties={failedJobSize=0, suspendedJobSize=1, totalJobSize=1}} in job com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines

com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31)

        at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30)

        at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>)

        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)

        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)

        at com.codahale.metrics.Timer.time(Timer.java:101)

        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)

        at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)

        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)

        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)

        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d.runUnchecked(<generated>)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)

        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)

        at com.github.rholder.retry.Retryer.call(Retryer.java:160)

        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)

        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)

        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:748)

[2017-07-12 07:28:11.523 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104' failed

        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d

        at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:1

 

[2017-07-12 07:28:11.531 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104': RUNNING -> ERROR

[2017-07-12 07:28:11.551 +0000] INFO  [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.d.DeploymentRepositoryService: Deployment 'Alok-test': BOOTSTRAPPING -> BOOTSTRAP_FAILED

[2017-07-12 07:28:14.056 +0000] INFO  [qtp802255644-15] GET /api/v8/environments/Alok-test/deployments/Alok-test/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL

[2017-07-12 07:28:14.056 +0000] INFO  [qtp802255644-15] GET /api/v8/environments/Alok-test/deployments/Alok-test/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE

Cloudera Employee
Posts: 49
Registered: ‎10-14-2014

Re: Clusters cannot be created with a failed/terminated Cloudera Manager.

It looks as though you're specifying the incorrect credentials for the instance that you're using. Often the intended initial username can differ with the AMI, e.g. many CentOS images use "centos" as the username, RHEL often use "ec2-user", etc. Are you using EC2 for this deployment? If so, what AMI are you using and on what region?

Explorer
Posts: 10
Registered: ‎07-12-2017

Re: Clusters cannot be created with a failed/terminated Cloudera Manager.

Hi Mike,

 

Thanks for your quick response.

 

I am using Centos 6.5 and the region on AWS i choosen is Singapore (ap-southeast-1b).

Yes i am using "ec2-user" as username. However on command line i am using "root" as user login name.

 

Please suggest, may i try using username as "root" and try to install.

 

Regards

Alok

Argyle Team

Cloudera Employee
Posts: 49
Registered: ‎10-14-2014

Re: Clusters cannot be created with a failed/terminated Cloudera Manager.

Yes, so long as you're able to login with it, you can specify it within Director. We'll allow you to specify any arbitrary username to make sure we cover as many edge cases as possible. :-) Let me know if this works for you!
Explorer
Posts: 10
Registered: ‎07-12-2017

Re: Clusters cannot be created with a failed/terminated Cloudera Manager.

Hi Mike,

 

As per the instruction i tried to setup, but still no luc and getting error "Boot Strap Failed"

 

Logs:-

 

[2017-07-18 06:43:55.580 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)
        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)
        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)
        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)
        ... 34 common frames omitted
[2017-07-18 06:44:08.351 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)
        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)
        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)
        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)
        ... 34 common frames omitted
[2017-07-18 06:44:28.352 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.19.72, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ...
[2017-07-18 06:44:28.352 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection.
[2017-07-18 06:44:28.372 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.19.72 was automatically accepted
[2017-07-18 06:44:28.436 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-19-72.ap-southeast-1.compute.internal was automatically accepted
[2017-07-18 06:44:28.499 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 54.169.68.43 was automatically accepted
[2017-07-18 06:44:28.562 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-54-169-68-43.ap-southeast-1.compute.amazonaws.com was automatically accepted
[2017-07-18 06:44:29.605 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2)
[2017-07-18 06:44:29.625 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.19.72 was automatically accepted
[2017-07-18 06:44:29.688 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-19-72.ap-southeast-1.compute.internal was automatically accepted
[2017-07-18 06:44:29.751 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 54.169.68.43 was automatically accepted
[2017-07-18 06:44:29.813 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-54-169-68-43.ap-southeast-1.compute.amazonaws.com was automatically accepted
[2017-07-18 06:44:30.856 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3)
[2017-07-18 06:44:30.876 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.19.72 was automatically accepted
[2017-07-18 06:44:30.938 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-19-72.ap-southeast-1.compute.internal was automatically accepted
[2017-07-18 06:44:31.001 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 54.169.68.43 was automatically accepted
[2017-07-18 06:44:31.065 +0000] WARN  [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-54-169-68-43.ap-southeast-1.compute.amazonaws.com was automatically accepted
[2017-07-18 06:44:31.111 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)
        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)
        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)
        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)
        ... 34 common frames omitted
[2017-07-18 06:45:03.892 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)
        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)
        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)
        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)
        ... 34 common frames omitted
[2017-07-18 06:45:03.893 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b suspended due to failure working on com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation
java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at com.google.common.base.Throwables.propagate(Throwables.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357)
        at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
        at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
        at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340)
        at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359)
        at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272)
        at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245)
        at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181)
        at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176)
        ... 34 common frames omitted
[2017-07-18 06:45:03.896 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b' failed
        at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a
        at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:0

[2017-07-18 06:45:03.896 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b': RUNNING -> SUSPENDED
[2017-07-18 06:45:04.223 +0000] INFO  [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Interrupting runner thread for pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b'
[2017-07-18 06:45:04.223 +0000] INFO  [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b' has already completed
[2017-07-18 06:45:04.240 +0000] INFO  [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b': SUSPENDED -> CANCELED
[2017-07-18 06:45:04.532 +0000] ERROR [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78)
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31)
        at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30)
        at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$be60b702.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
[2017-07-18 06:45:04.535 +0000] ERROR [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error ErrorInfo{code=ErrorCode{name=JOB_EXECUTION_FAIL, type=SERVICE, retryable=true, keys=[failedJobSize, suspendedJobSize, totalJobSize]}, properties={failedJobSize=0, suspendedJobSize=1, totalJobSize=1}} in job com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines
com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78)
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31)
        at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30)
        at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67)
        at com.codahale.metrics.Timer.time(Timer.java:101)
        at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63)
        at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
        at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
        at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$be60b702.runUnchecked(<generated>)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168)
        at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
        at com.github.rholder.retry.Retryer.call(Retryer.java:160)
        at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133)
        at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157)
        at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
[2017-07-18 06:45:04.535 +0000] ERROR [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff' failed
        at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$be60b702
        at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:1

[2017-07-18 06:45:04.540 +0000] INFO  [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.s.PipelineRepositoryService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff': RUNNING -> ERROR
[2017-07-18 06:45:04.559 +0000] INFO  [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.d.DeploymentRepositoryService: Deployment 'testing': BOOTSTRAPPING -> BOOTSTRAP_FAILED
[2017-07-18 06:45:05.293 +0000] INFO  [qtp802255644-119] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL
[2017-07-18 06:45:05.293 +0000] INFO  [qtp802255644-119] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE
[2017-07-18 06:45:10.297 +0000] INFO  [qtp802255644-14] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL
[2017-07-18 06:45:10.297 +0000] INFO  [qtp802255644-14] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE
[2017-07-18 06:45:15.295 +0000] INFO  [qtp802255644-115] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL
[2017-07-18 06:45:15.295 +0000] INFO  [qtp802255644-115] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE
[2017-07-18 06:45:20.327 +0000] INFO  [qtp802255644-16] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL
[2017-07-18 06:45:20.327 +0000] INFO  [qtp802255644-16] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE
[2017-07-18 06:45:23.724 +0000] INFO  [task-thread-10] - - - - c.c.launchpad.task.RefreshClusters: Refreshing Cluster models
[2017-07-18 06:45:23.725 +0000] INFO  [task-thread-10] - - - - c.c.launchpad.task.RefreshClusters: Finished refreshing all pre-existing Cluster models
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Refreshing pre-existing Deployments
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:Argyle as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:Alok as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:ALok as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:test as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment alok14:alok-alok as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Alok-test:testing as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true)
[2017-07-18 06:45:24.357 +0000] INFO  [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Finished refreshing all pre-existing Deployment models
[2017-07-18 06:45:25.307 +0000] INFO  [qtp802255644-19] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL
[2017-07-18 06:45:25.308 +0000] INFO  [qtp802255644-19] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE

Highlighted
Cloudera Employee
Posts: 49
Registered: ‎10-14-2014

Re: Clusters cannot be created with a failed/terminated Cloudera Manager.

It looks as though the user authentication is still the problem. Can you try provisioning an instance of the AMI you're trying to use manually and attempting to connect using the credentials you're providing to Director?

i.e. Launch an instance using the EC2 console and just attempting to SSH into it.
Explorer
Posts: 10
Registered: ‎07-12-2017

Re: Clusters cannot be created with a failed/terminated Cloudera Manager.

Mike,

 

I already checked the same and able to ssh the instance.

 

Regards

Alok Katiyar

Cloudera Employee
Posts: 49
Registered: ‎10-14-2014

Re: Clusters cannot be created with a failed/terminated Cloudera Manager.

Okay, hrm. Are you using the web UI, or a conf file to set up this cluster? If you're using a conf file, would you be able to post it on something like pastebin (being sure to redact anything sensitive like AWS secret keys, access keys, and SSH keys) and link it here? Also would you be able to give me the exact AMI ID that's being used?

Are you certain you're using the same private key when trying to set up a cluster that you're using to SSH into raw instances? This error is still consistent with incorrect SSH credentials being supplied -- we just have to figure out exactly what's going on here.
Explorer
Posts: 10
Registered: ‎07-12-2017

Re: Clusters cannot be created with a failed/terminated Cloudera Manager.

Hello Mike,

 

I am able to resolve the issue and able to deploy Cloudera Manager, I am into more troubleshooting and will catch you soon with more questions..lolz..

 

Hope you dont mind. Thanks for the support.

 

Regards

Alok

Explorer
Posts: 10
Registered: ‎07-12-2017

Re: Clusters cannot be created with a failed/terminated Cloudera Manager.

Hello Mike,

 

Require small information, when i turn off the instances and start again my cluster is not coming up, moreover my Cloudera Director is working fine but the cluster made under enviorment is done bad and my instances (Master and worker ) are not coming up and i have to Terminate the cluster and need to re-create again.

 

Also, i am unable to add services and getting error for "cdh" i tried to install "cdh" but it stuck to an point.

 

Kindly interven and require your inputs.

 

Regards

Alok

Announcements