Member since
07-12-2017
10
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 8835 | 07-27-2017 10:43 PM |
07-29-2017
05:27 AM
Hi All, I am able to completed Cloudera Manager setup (Ver: 5.12) and able to complete CDH (5.12) installation. But i am unable to download Accumulo and Presto parcel on cloudera manager. Please provide Accumulo and Presto parcel path to download and install on CM.
... View more
07-27-2017
10:43 PM
CDH installation stucks # cat cloudera-scm-server.log | grep cdh 2017-07-27 22:27:59,881 INFO 1952372338@scm-web-26:com.cloudera.parcel.components.ParcelManagerImpl: Downloading parcels of CDH:5.12.0-1.cdh5.12.0.p0.29 for distros [RHEL7] 2017-07-27 22:27:59,901 INFO 1952372338@scm-web-26:com.cloudera.parcel.components.ParcelDownloaderImpl: Preparing to download: http://archive.cloudera.com/cdh5/parcels/5.12/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel 2017-07-27 22:28:26,159 INFO New I/O worker #5:com.cloudera.parcel.components.ParcelDownloaderImpl: Completed download of: http://archive.cloudera.com/cdh5/parcels/5.12/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel 2017-07-27 22:28:36,354 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Discovered parcel on CM server: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel 2017-07-27 22:28:36,355 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Created torrent file: /opt/cloudera/parcel-repo/CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent 2017-07-27 22:28:36,358 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Creating single-file torrent for CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel... 2017-07-27 22:28:36,359 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashing data from CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel with 2 threads (3248 pieces)... 2017-07-27 22:28:45,278 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Torrent name: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel 2017-07-27 22:28:50,292 INFO 56131067@scm-web-36:com.cloudera.parcel.components.ParcelManagerImpl: Distributing parcel CDH:5.12.0-1.cdh5.12.0.p0.29 on cluster Cluster 2017-07-27 22:28:50,618 WARN 831920394@scm-web-28:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent 2017-07-27 22:28:50,665 INFO 831920394@scm-web-28:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.24.106 over PT0.047S. Rate: 1.321 MB/s 2017-07-27 22:28:50,635 WARN 2073794887@scm-web-37:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent 2017-07-27 22:28:50,668 WARN 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: No hash for parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent 2017-07-27 22:28:50,681 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.27.115 over PT0.013S. Rate: 4.778 MB/s 2017-07-27 22:28:50,706 INFO 2073794887@scm-web-37:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel.torrent (65127 bytes) to 172.31.23.5 over PT0.039S. Rate: 1.593 MB/s 2017-07-27 22:28:50,884 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Parcel download request: CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel from : 172.31.24.106 2017-07-27 22:28:50,908 INFO 266846999@scm-web-33:com.cloudera.server.web.cmf.ParcelController: Served parcel piece CDH-5.12.0-1.cdh5.12.0.p0.29-el7.parcel (1048576 bytes) to 172.31.24.106 over PT0.012S. Rate: 83.333 MB/s 2017-07-27 22:32:25,227 INFO 430900238@scm-web-40:com.cloudera.parcel.components.ParcelManagerImpl: Activating parcel CDH:5.12.0-1.cdh5.12.0.p0.29 on cluster Cluster 2017-07-27 22:32:25,261 INFO 430900238@scm-web-40:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command ProcessStalenessCheckCommand BasicCmdArgs{args=[First reason why: com.cloudera.cmf.model.DbCluster.cdhVersionForDb (#1) has changed]}. Also, i have checked full logs without grep and found no error. Please suggest.
... View more
07-27-2017
02:03 AM
Hi Team, I am facing issue while CDH installation process on Cloudera director. 1) On cloudera director getting error "Bootstrap Failed" while installtion of CDH 2) When i am trying to install CDH thru parcel on Cloudera Manager GUI, there it stucks at Distributing and not going forward. 3) I check and tried with same version for Cloudera Manager and CDH, but still CDH installation do go further. Logs :- [2017-07-27 08:42:29.156 +0000] INFO [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null} [2017-07-27 08:42:31.181 +0000] INFO [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null} [2017-07-27 08:42:33.203 +0000] INFO [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null} [2017-07-27 08:42:35.224 +0000] INFO [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null} [2017-07-27 08:42:37.246 +0000] INFO [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null} [2017-07-27 08:42:39.268 +0000] INFO [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null} [2017-07-27 08:42:41.290 +0000] INFO [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null} [2017-07-27 08:42:43.314 +0000] INFO [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null} [2017-07-27 08:42:45.337 +0000] INFO [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null} [2017-07-27 08:42:47.360 +0000] INFO [p-8448dc82972c-DefaultBootstrapClusterJob] 0085ebef-d791-477a-93a8-e033ae82c913 POST /api/v9/environments/Test-2/deployments/Cloudera%20Manager/clusters com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.12.0-1.cdh5.12.0.p0.29) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=900, count=0, countTotal=3, warnings=null, errors=null}
... View more
07-24-2017
04:37 AM
Hi Team, If we want to change the password for Cloudera Director, what will be procedure to do it on command line Regards Alok
... View more
07-20-2017
06:58 AM
Hello Mike, Require small information, when i turn off the instances and start again my cluster is not coming up, moreover my Cloudera Director is working fine but the cluster made under enviorment is done bad and my instances (Master and worker ) are not coming up and i have to Terminate the cluster and need to re-create again. Also, i am unable to add services and getting error for "cdh" i tried to install "cdh" but it stuck to an point. Kindly interven and require your inputs. Regards Alok
... View more
07-19-2017
09:40 PM
Hello Mike, I am able to resolve the issue and able to deploy Cloudera Manager, I am into more troubleshooting and will catch you soon with more questions..lolz.. Hope you dont mind. Thanks for the support. Regards Alok
... View more
07-18-2017
09:50 PM
Mike, I already checked the same and able to ssh the instance. Regards Alok Katiyar
... View more
07-18-2017
02:48 AM
Hi Mike, As per the instruction i tried to setup, but still no luc and getting error "Boot Strap Failed" Logs:- [2017-07-18 06:43:55.580 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at com.google.common.base.Throwables.propagate(Throwables.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357) at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280) at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245) at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176) ... 34 common frames omitted [2017-07-18 06:44:08.351 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at com.google.common.base.Throwables.propagate(Throwables.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357) at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280) at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245) at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176) ... 34 common frames omitted [2017-07-18 06:44:28.352 +0000] INFO [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.19.72, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ... [2017-07-18 06:44:28.352 +0000] INFO [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection. [2017-07-18 06:44:28.372 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.19.72 was automatically accepted [2017-07-18 06:44:28.436 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-19-72.ap-southeast-1.compute.internal was automatically accepted [2017-07-18 06:44:28.499 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 54.169.68.43 was automatically accepted [2017-07-18 06:44:28.562 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-54-169-68-43.ap-southeast-1.compute.amazonaws.com was automatically accepted [2017-07-18 06:44:29.605 +0000] INFO [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2) [2017-07-18 06:44:29.625 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.19.72 was automatically accepted [2017-07-18 06:44:29.688 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-19-72.ap-southeast-1.compute.internal was automatically accepted [2017-07-18 06:44:29.751 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 54.169.68.43 was automatically accepted [2017-07-18 06:44:29.813 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-54-169-68-43.ap-southeast-1.compute.amazonaws.com was automatically accepted [2017-07-18 06:44:30.856 +0000] INFO [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3) [2017-07-18 06:44:30.876 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.19.72 was automatically accepted [2017-07-18 06:44:30.938 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-19-72.ap-southeast-1.compute.internal was automatically accepted [2017-07-18 06:44:31.001 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 54.169.68.43 was automatically accepted [2017-07-18 06:44:31.065 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-54-169-68-43.ap-southeast-1.compute.amazonaws.com was automatically accepted [2017-07-18 06:44:31.111 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at com.google.common.base.Throwables.propagate(Throwables.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357) at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280) at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245) at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176) ... 34 common frames omitted [2017-07-18 06:45:03.892 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at com.google.common.base.Throwables.propagate(Throwables.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357) at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280) at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245) at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176) ... 34 common frames omitted [2017-07-18 06:45:03.893 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b suspended due to failure working on com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at com.google.common.base.Throwables.propagate(Throwables.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357) at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280) at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245) at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176) ... 34 common frames omitted [2017-07-18 06:45:03.896 +0000] ERROR [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b' failed at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$e1c2329a at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:0 [2017-07-18 06:45:03.896 +0000] INFO [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b': RUNNING -> SUSPENDED [2017-07-18 06:45:04.223 +0000] INFO [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Interrupting runner thread for pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b' [2017-07-18 06:45:04.223 +0000] INFO [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b' has already completed [2017-07-18 06:45:04.240 +0000] INFO [p-3bb25943251b-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff/child-00000-3be673e8-c7aa-4be0-b5f9-3bb25943251b': SUSPENDED -> CANCELED [2017-07-18 06:45:04.532 +0000] ERROR [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0 at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78) at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31) at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30) at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$be60b702.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) [2017-07-18 06:45:04.535 +0000] ERROR [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error ErrorInfo{code=ErrorCode{name=JOB_EXECUTION_FAIL, type=SERVICE, retryable=true, keys=[failedJobSize, suspendedJobSize, totalJobSize]}, properties={failedJobSize=0, suspendedJobSize=1, totalJobSize=1}} in job com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0 at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78) at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31) at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30) at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor196.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$be60b702.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) [2017-07-18 06:45:04.535 +0000] ERROR [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff' failed at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$be60b702 at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:1 [2017-07-18 06:45:04.540 +0000] INFO [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.s.PipelineRepositoryService: Pipeline 'f6bb21e1-7bf0-4b28-aa5f-e8fefef3beff': RUNNING -> ERROR [2017-07-18 06:45:04.559 +0000] INFO [p-e8fefef3beff-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.d.DeploymentRepositoryService: Deployment 'testing': BOOTSTRAPPING -> BOOTSTRAP_FAILED [2017-07-18 06:45:05.293 +0000] INFO [qtp802255644-119] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL [2017-07-18 06:45:05.293 +0000] INFO [qtp802255644-119] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE [2017-07-18 06:45:10.297 +0000] INFO [qtp802255644-14] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL [2017-07-18 06:45:10.297 +0000] INFO [qtp802255644-14] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE [2017-07-18 06:45:15.295 +0000] INFO [qtp802255644-115] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL [2017-07-18 06:45:15.295 +0000] INFO [qtp802255644-115] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE [2017-07-18 06:45:20.327 +0000] INFO [qtp802255644-16] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL [2017-07-18 06:45:20.327 +0000] INFO [qtp802255644-16] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE [2017-07-18 06:45:23.724 +0000] INFO [task-thread-10] - - - - c.c.launchpad.task.RefreshClusters: Refreshing Cluster models [2017-07-18 06:45:23.725 +0000] INFO [task-thread-10] - - - - c.c.launchpad.task.RefreshClusters: Finished refreshing all pre-existing Cluster models [2017-07-18 06:45:24.357 +0000] INFO [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Refreshing pre-existing Deployments [2017-07-18 06:45:24.357 +0000] INFO [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:Argyle as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true) [2017-07-18 06:45:24.357 +0000] INFO [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:Alok as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true) [2017-07-18 06:45:24.357 +0000] INFO [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:ALok as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true) [2017-07-18 06:45:24.357 +0000] INFO [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Argyle:test as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true) [2017-07-18 06:45:24.357 +0000] INFO [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment alok14:alok-alok as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true) [2017-07-18 06:45:24.357 +0000] INFO [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Skipping refresh of deployment Alok-test:testing as it is in transition or terminated. (Stage: BOOTSTRAP_FAILED, deploymentIsNull: true) [2017-07-18 06:45:24.357 +0000] INFO [task-thread-2] - - - - c.c.l.task.RefreshDeployments: Finished refreshing all pre-existing Deployment models [2017-07-18 06:45:25.307 +0000] INFO [qtp802255644-19] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL [2017-07-18 06:45:25.308 +0000] INFO [qtp802255644-19] GET /api/v8/environments/Alok-test/deployments/testing/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE
... View more
07-12-2017
08:00 AM
Hi Mike, Thanks for your quick response. I am using Centos 6.5 and the region on AWS i choosen is Singapore (ap-southeast-1b). Yes i am using "ec2-user" as username. However on command line i am using "root" as user login name. Please suggest, may i try using username as "root" and try to install. Regards Alok Argyle Team
... View more
07-12-2017
01:21 AM
Clusters cannot be created with a failed/terminated Cloudera Manager. Logs:- [2017-07-12 07:26:49.967 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at com.google.common.base.Throwables.propagate(Throwables.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357) at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280) at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245) at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176) ... 34 common frames omitted [2017-07-12 07:26:59.968 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.21.84, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ... [2017-07-12 07:26:59.968 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection. [2017-07-12 07:26:59.988 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted [2017-07-12 07:27:00.052 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted [2017-07-12 07:27:00.115 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted [2017-07-12 07:27:00.178 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted [2017-07-12 07:27:01.221 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2) [2017-07-12 07:27:01.240 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted [2017-07-12 07:27:01.303 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted [2017-07-12 07:27:01.366 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted [2017-07-12 07:27:01.433 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted [2017-07-12 07:27:02.476 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3) [2017-07-12 07:27:02.497 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted [2017-07-12 07:27:02.559 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted [2017-07-12 07:27:02.623 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted [2017-07-12 07:27:02.685 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted [2017-07-12 07:27:38.243 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at com.google.common.base.Throwables.propagate(Throwables.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357) at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280) at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245) at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176) ... 34 common frames omitted [2017-07-12 07:28:05.415 +0000] INFO [task-thread-5] - - - - c.c.l.m.r.DeploymentsReporter: Enqueueing all deployments for usage reporting [2017-07-12 07:28:05.418 +0000] INFO [task-thread-5] - - - - c.c.l.m.r.DeploymentsReporter: Enqueueing 0 deployments for usage reporting [2017-07-12 07:28:08.243 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: >> AllocateInstances$WaitForSshCredentialInstallation/2 [PluggableComputeInstance{ipAddress=172.31.21.84, delegate=null, hostEndpoints=[HostEndpoint{hostAdd ... [2017-07-12 07:28:08.243 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Attempting SSH connection. [2017-07-12 07:28:08.263 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted [2017-07-12 07:28:08.326 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted [2017-07-12 07:28:08.390 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted [2017-07-12 07:28:08.456 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted [2017-07-12 07:28:09.499 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 2) [2017-07-12 07:28:09.518 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted [2017-07-12 07:28:09.627 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted [2017-07-12 07:28:09.688 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted [2017-07-12 07:28:09.751 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted [2017-07-12 07:28:10.794 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.cloudera.launchpad.sshj.SshJClient: Retrying SSH connection (attempt 3) [2017-07-12 07:28:10.813 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 172.31.21.84 was automatically accepted [2017-07-12 07:28:10.876 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ip-172-31-21-84.ap-southeast-1.compute.internal was automatically accepted [2017-07-12 07:28:10.937 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for 52.77.211.171 was automatically accepted [2017-07-12 07:28:11.000 +0000] WARN [reader] - - - - c.c.l.sshj.TrustAnyHostKeyVerifier: Host key for ec2-52-77-211-171.ap-southeast-1.compute.amazonaws.com was automatically accepted [2017-07-12 07:28:11.043 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at com.google.common.base.Throwables.propagate(Throwables.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357) at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280) at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245) at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176) ... 34 common frames omitted [2017-07-12 07:28:11.043 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d suspended due to failure working on com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation java.lang.RuntimeException: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at com.google.common.base.Throwables.propagate(Throwables.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:188) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:388) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation.run(AllocateInstances.java:357) at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods at net.schmizz.sshj.SSHClient.auth(SSHClient.java:225) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:340) at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:359) at com.cloudera.launchpad.sshj.SshJClient.applyCredentials(SshJClient.java:313) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:280) at com.cloudera.launchpad.sshj.SshJClient.attemptConnectionWithOptionalTunneling(SshJClient.java:272) at com.cloudera.launchpad.sshj.SshJClient.attemptConnection(SshJClient.java:245) at com.cloudera.launchpad.sshj.SshJClient.access$000(SshJClient.java:54) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:181) at com.cloudera.launchpad.sshj.SshJClient$1.call(SshJClient.java:176) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.sshj.SshJClient.connect(SshJClient.java:176) ... 34 common frames omitted [2017-07-12 07:28:11.044 +0000] ERROR [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.DatabasePipelineRunner: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d' failed at com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation$$EnhancerBySpringCGLIB$$7a4fafd5 at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:0 [2017-07-12 07:28:11.044 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d': RUNNING -> SUSPENDED [2017-07-12 07:28:11.200 +0000] INFO [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Interrupting runner thread for pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d' [2017-07-12 07:28:11.200 +0000] INFO [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d' has already completed [2017-07-12 07:28:11.207 +0000] INFO [p-59c3b14db89d-WaitForSshCredentialInstallation] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.bootstrap.AllocateInstances$WaitForSshCredentialInstallation - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104/child-00000-3a65ebd2-a1ef-4ac6-b995-59c3b14db89d': SUSPENDED -> CANCELED [2017-07-12 07:28:11.522 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0 at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78) at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31) at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30) at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) [2017-07-12 07:28:11.523 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error ErrorInfo{code=ErrorCode{name=JOB_EXECUTION_FAIL, type=SERVICE, retryable=true, keys=[failedJobSize, suspendedJobSize, totalJobSize]}, properties={failedJobSize=0, suspendedJobSize=1, totalJobSize=1}} in job com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: Parallel execution of jobs failed. Total: 1 Canceled: 1 Failed: 0 at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:78) at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines.run(UnboundedWaitForAllPipelines.java:31) at com.cloudera.launchpad.pipeline.job.Job1.runUnchecked(Job1.java:30) at com.cloudera.launchpad.pipeline.job.Job1$$FastClassBySpringCGLIB$$54178501.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) at com.codahale.metrics.Timer.time(Timer.java:101) at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656) at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d.runUnchecked(<generated>) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:197) at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:168) at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) at com.github.rholder.retry.Retryer.call(Retryer.java:160) at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:133) at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:157) at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) [2017-07-12 07:28:11.523 +0000] ERROR [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.DatabasePipelineRunner: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104' failed at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$56ee343d at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:1 [2017-07-12 07:28:11.531 +0000] INFO [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.p.s.PipelineRepositoryService: Pipeline 'c9469876-cb54-46ef-8aea-7548aa437104': RUNNING -> ERROR [2017-07-12 07:28:11.551 +0000] INFO [p-7548aa437104-DefaultBootstrapDeploymentJob] POST /api/v8/environments/Alok-test/deployments com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines - c.c.l.d.DeploymentRepositoryService: Deployment 'Alok-test': BOOTSTRAPPING -> BOOTSTRAP_FAILED [2017-07-12 07:28:14.056 +0000] INFO [qtp802255644-15] GET /api/v8/environments/Alok-test/deployments/Alok-test/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal error code JOB_EXECUTION_FAIL [2017-07-12 07:28:14.056 +0000] INFO [qtp802255644-15] GET /api/v8/environments/Alok-test/deployments/Alok-test/status - - c.c.l.api.v8.model.error.ErrorInfoV8: Converting internal exception type: SERVICE
... View more
Labels:
- Labels:
-
Cloudera Manager