Cloudera Director 2.0 - Specify GCP subnet in template definition

morganstake — Thu, 10 Mar 2016 02:00:17 GMT

Hello,

I have a GCP network with 2 subnets.

I am trying to define a template in the Cloudera Director to create the VM in a subnet, but the UI will only allow the parent network, without any indication of how to specify the subnetwork.

The network looks like so:

parentNet

--> subnet1

--> subnet2

The UI only allows me to specify the parentNet, specifying anything else gets:

Ultimately, the message is:

"Invalid value for field 'resource.networkInterfaces[0].subnetwork': ''. Subnetwork was not specified for a custom network."

This fails with the following trace:

[2016-03-10 01:40:27] ERROR [pipeline-thread-3] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

com.cloudera.director.spi.v1.model.exception.UnrecoverableProviderException: Problem allocating instances.

at com.cloudera.director.google.compute.GoogleComputeProvider.allocate(GoogleComputeProvider.java:388) ~[na:na]

at com.cloudera.director.google.compute.GoogleComputeProvider.allocate(GoogleComputeProvider.java:85) ~[na:na]

at com.cloudera.launchpad.pluggable.compute.PluggableComputeProvider.allocate(PluggableComputeProvider.java:492) ~[launchpad-pluggable-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pluggable.compute.PluggableComputeProvider.allocateInstancesForTemplate(PluggableComputeProvider.java:441) ~[launchpad-pluggable-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pluggable.compute.PluggableComputeProvider.allocate(PluggableComputeProvider.java:419) ~[launchpad-pluggable-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pluggable.compute.PluggableComputeProvider.allocate(PluggableComputeProvider.java:267) ~[launchpad-pluggable-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.bootstrap.AllocateInstances$AllocateAndWaitForInstancesToRun.run(AllocateInstances.java:153) ~[launchpad-bootstrap-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.bootstrap.AllocateInstances$AllocateAndWaitForInstancesToRun.run(AllocateInstances.java:133) ~[launchpad-bootstrap-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) ~[launchpad-pipeline-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) ~[spring-core-4.1.6.RELEASE.jar!/:2.0.0]

at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) ~[launchpad-pipeline-2.0.0.jar!/:2.0.0]

at com.codahale.metrics.Timer.time(Timer.java:101) ~[metrics-core-3.1.0.jar!/:3.1.0]

at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) ~[launchpad-pipeline-2.0.0.jar!/:2.0.0]

at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source) ~[na:na]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_73]

at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_73]

at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at com.cloudera.launchpad.bootstrap.AllocateInstances$AllocateAndWaitForInstancesToRun$$EnhancerBySpringCGLIB$$e5756658.runUnchecked(<generated>) ~[spring-core-4.1.6.RELEASE.jar!/:2.0.0]

at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:159) [launchpad-pipeline-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:130) [launchpad-pipeline-2.0.0.jar!/:2.0.0]

at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) [guava-retrying-1.0.6.jar!/:na]

at com.github.rholder.retry.Retryer.call(Retryer.java:110) [guava-retrying-1.0.6.jar!/:na]

at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:99) [launchpad-pipeline-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:125) [launchpad-pipeline-database-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) [launchpad-common-2.0.0.jar!/:2.0.0]

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_73]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_73]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_73]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_73]

at java.lang.Thread.run(Thread.java:745) [na:1.8.0_73]

Suppressed: com.cloudera.launchpad.pluggable.common.ExceptionConditions$DetailHolderException: Exception details:

key: null

PluginExceptionCondition{type=ERROR, message='400 Bad Request

{

"code" : 400,

"errors" : [ {

"domain" : "global",

"message" : "Invalid value for field 'resource.networkInterfaces[0].subnetwork': ''. Subnetwork was not specified for a custom network.",

"reason" : "invalid"

} ],

"message" : "Invalid value for field 'resource.networkInterfaces[0].subnetwork': ''. Subnetwork was not specified for a custom network."

}'}

[2016-03-10 01:40:27] ERROR [pipeline-thread-3] - c.c.l.pipeline.util.PipelineRunner: (Suppressed)

com.cloudera.launchpad.pluggable.common.ExceptionConditions$DetailHolderException: Exception details:

key: null

PluginExceptionCondition{type=ERROR, message='400 Bad Request

{

"code" : 400,

"errors" : [ {

"domain" : "global",

"message" : "Invalid value for field 'resource.networkInterfaces[0].subnetwork': ''. Subnetwork was not specified for a custom network.",

"reason" : "invalid"

} ],

"message" : "Invalid value for field 'resource.networkInterfaces[0].subnetwork': ''. Subnetwork was not specified for a custom network."

}'}

Thank you in advance.

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

jadair — Thu, 10 Mar 2016 20:58:41 GMT

Per the Google Cloud Platform documentation on Using Subnetworks, there are two kinds of networking in Google Cloud Platform, the original Legacy (non-subnet) Networks, and the newer Subnet Networks. From the description in your post, I assume you are using Subnet Networks. The Google plugin for Cloudera Director currently supports only Legacy (non-subnet) Networks. I have filed an enhancement request issue (#118) against the plugin, and made sure developers at Cloudera and Google are aware of the issue.

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

morganstake — Thu, 10 Mar 2016 21:03:19 GMT

I saw that and I was hoping I read that wrong.
I'm available for beta testing if that helps move it along.

Thank you for the quick reply!

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

jadair — Thu, 10 Mar 2016 21:18:42 GMT

I recommend keeping an eye on that issue in the plugin repository. If a fix appears there, you should be able to try out a local build of the plugin in your Cloudera Director installation before we ship an updated version of the plugin in a future Cloudera Director release. I will try to comment here if I hear anything about fix versions.

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

morganstake — Thu, 10 Mar 2016 21:21:15 GMT

I just cloned the repo and I'm looking through the code to see if I can do it myself.

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

yarivgraf — Thu, 08 Nov 2018 12:51:02 GMT

Any news regarding GCP legacy network and subnetwork?
Last time I checked (a few months ago) it was impossible to deploy in a subnetwork.

Thanks

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

Bill Havanki — Thu, 08 Nov 2018 13:14:13 GMT

Hello yarivgraf,

Good news: A community member implemented subnetwork support in the Google plugin, and the work was merged several weeks ago.

https://github.com/cloudera/director-google-plugin/pull/150

The next 6.x release of Cloudera Altus Director will come packaged with a new plugin release that includes this change. In the meantime, you can build the plugin and install it into your existing Altus Director 2.8 or 6.0 installations and it should work.

* For Director 2.8, build and install the plugin from the v1.x branch.

* For Director 6.0, build and install the plugin from the v2.0.x branch.

The plugin's README describes the build process and links to some docs on installing the plugin in Altus Director. For the latter, you basically place the plugin JAR into Altus Director's plugins directory, replacing the prior Google plugin JAR.

question Re: Cloudera Director 2.0 - Specify GCP subnet in template definition in Archives of Support Questions (Read Only)

Cloudera Director 2.0 - Specify GCP subnet in template definition

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition

Re: Cloudera Director 2.0 - Specify GCP subnet in template definition