Reply
Highlighted
New Contributor
Posts: 11
Registered: ‎02-13-2014
Accepted Solution

Cloudera Director: API call to Cloudera Manager failed error

When building a new Cluster using Cloudera Director 1.5.1 it fails with the error: "API call to Cloudera Manager failed."

 

I've experienced the problem 3 times in a row. First when creating Manager and Cluster at the same time, then 2 times when the Manager is already up and running.

 

I haven't got any other error before this problem, which I'm guessing is one of the latest steps to have a working cluster.

 

Stacktrace is below.

 

ERROR [pipeline-thread-80] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
com.cloudera.api.ext.ClouderaManagerException: API call to Cloudera Manager failed. Method=ApiRootResource.getCurrentVersion, Args=null
	at com.cloudera.api.ext.ClouderaManagerClientProxy.invoke(ClouderaManagerClientProxy.java:79) ~[launchpad-cloudera-manager-api-ext-1.5.1.jar!/:na]
	at com.sun.proxy.$Proxy170.getCurrentVersion(Unknown Source) ~[na:na]
	at com.cloudera.launchpad.bootstrap.cluster.BootstrapClouderaManagerAgent$HostInstall.run(BootstrapClouderaManagerAgent.java:145) ~[launchpad-bootstrap-1.5.1.jar!/:1.5.1]
	at com.cloudera.launchpad.bootstrap.cluster.BootstrapClouderaManagerAgent$HostInstall.run(BootstrapClouderaManagerAgent.java:110) ~[launchpad-bootstrap-1.5.1.jar!/:1.5.1]
	at com.cloudera.launchpad.pipeline.job.Job3.runUnchecked(Job3.java:32) ~[launchpad-pipeline-1.5.1.jar!/:1.5.1]
	at com.cloudera.launchpad.pipeline.job.Job3$$FastClassBySpringCGLIB$$54178503.invoke(<generated>) ~[spring-core-4.1.5.RELEASE.jar!/:1.5.1]
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717) ~[spring-aop-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) ~[spring-aop-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:55) ~[launchpad-pipeline-1.5.1.jar!/:1.5.1]
	at com.codahale.metrics.Timer.time(Timer.java:101) ~[metrics-core-3.1.0.jar!/:3.1.0]
	at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:51) ~[launchpad-pipeline-1.5.1.jar!/:1.5.1]
	at sun.reflect.GeneratedMethodAccessor143.invoke(Unknown Source) ~[na:na]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_65]
	at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) ~[spring-aop-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) ~[spring-aop-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) ~[spring-aop-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) ~[spring-aop-4.1.5.RELEASE.jar!/:4.1.5.RELEASE]
	at com.cloudera.launchpad.bootstrap.cluster.BootstrapClouderaManagerAgent$HostInstall$$EnhancerBySpringCGLIB$$a9863238.runUnchecked(<generated>) ~[spring-core-4.1.5.RELEASE.jar!/:1.5.1]
	at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:165) [launchpad-pipeline-1.5.1.jar!/:1.5.1]
	at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:136) [launchpad-pipeline-1.5.1.jar!/:1.5.1]
	at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) [guava-retrying-1.0.6.jar!/:na]
	at com.github.rholder.retry.Retryer.call(Retryer.java:110) [guava-retrying-1.0.6.jar!/:na]
	at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:98) [launchpad-pipeline-1.5.1.jar!/:1.5.1]
	at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:120) [launchpad-pipeline-database-1.5.1.jar!/:1.5.1]
	at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) [launchpad-common-1.5.1.jar!/:1.5.1]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_65]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_65]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
Cloudera Employee
Posts: 119
Registered: ‎10-13-2014

Re: Cloudera Director: API call to Cloudera Manager failed error

What instance type are you using for Cloudera Manager and for the cluster nodes? What services does the cluster run and on how many instances? 

 

 

One reason why that API call may fail is if Cloudera Manager runs out of memory. Are you able to access Cloudera Manager with a browser after the failure? 

New Contributor
Posts: 11
Registered: ‎02-13-2014

Re: Cloudera Director: API call to Cloudera Manager failed error

Hi, instance type being used for the Manager is m3.medium and for the nodes I'm using m3.xlarge for all but Gateway, which also uses m3.medium.

 

I've picked the Hadoop with Impala option with no customization. I've noticed CM is not accesible, so probably it haven't started. I'll take a look on that.

 

Which instance type would you recommend for CM instead?

Cloudera Employee
Posts: 119
Registered: ‎10-13-2014

Re: Cloudera Director: API call to Cloudera Manager failed error

m3.medium is definitely too small to run Cloudera Manager and all the management services. 

 

My recommendation is to try again with m4.large or m4.xlarge as the Cloudera Manager instance. That should take memory related failures out of the picture. 

New Contributor
Posts: 11
Registered: ‎02-13-2014

Re: Cloudera Director: API call to Cloudera Manager failed error

Awesome. Thank you.