Cloudera Director 2.0 - Specify GCP subnet in template definition

[#1]

Hello,

I have a GCP network with 2 subnets.

I am trying to define a template in the Cloudera Director to create the VM in a subnet, but the UI will only allow the parent network, without any indication of how to specify the subnetwork.

The network looks like so:

parentNet

   |

   --> subnet1

   --> subnet2

The UI only allows me to specify the parentNet, specifying anything else gets:

--------------

Error processing request. Verify the provided values are correct and try again.

Parameter network must conform to the pattern [a-z](?:[-a-z0-9]{0,61}[a-z0-9])?

--------------

or when specifying the subnet1 in the network name:

--------------

Network 'subnet1' not found for project 'my-project-id-here'.

-------------

Ultimately, the message is:

"Invalid value for field 'resource.networkInterfaces[0].subnetwork': ''.  Subnetwork was not specified for a custom network."

This fails with the following trace:

[2016-03-10 01:40:27] ERROR [pipeline-thread-3] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed

com.cloudera.director.spi.v1.model.exception.UnrecoverableProviderException: Problem allocating instances.

at com.cloudera.director.google.compute.GoogleComputeProvider.allocate(GoogleComputeProvider.java:388) ~[na:na]

at com.cloudera.director.google.compute.GoogleComputeProvider.allocate(GoogleComputeProvider.java:85) ~[na:na]

at com.cloudera.launchpad.pluggable.compute.PluggableComputeProvider.allocate(PluggableComputeProvider.java:492) ~[launchpad-pluggable-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pluggable.compute.PluggableComputeProvider.allocateInstancesForTemplate(PluggableComputeProvider.java:441) ~[launchpad-pluggable-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pluggable.compute.PluggableComputeProvider.allocate(PluggableComputeProvider.java:419) ~[launchpad-pluggable-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pluggable.compute.PluggableComputeProvider.allocate(PluggableComputeProvider.java:267) ~[launchpad-pluggable-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.bootstrap.AllocateInstances$AllocateAndWaitForInstancesToRun.run(AllocateInstances.java:153) ~[launchpad-bootstrap-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.bootstrap.AllocateInstances$AllocateAndWaitForInstancesToRun.run(AllocateInstances.java:133) ~[launchpad-bootstrap-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31) ~[launchpad-pipeline-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>) ~[spring-core-4.1.6.RELEASE.jar!/:2.0.0]

at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) ~[launchpad-pipeline-2.0.0.jar!/:2.0.0]

at com.codahale.metrics.Timer.time(Timer.java:101) ~[metrics-core-3.1.0.jar!/:3.1.0]

at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) ~[launchpad-pipeline-2.0.0.jar!/:2.0.0]

at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source) ~[na:na]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_73]

at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_73]

at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) ~[spring-aop-4.1.6.RELEASE.jar!/:4.1.6.RELEASE]

at com.cloudera.launchpad.bootstrap.AllocateInstances$AllocateAndWaitForInstancesToRun$$EnhancerBySpringCGLIB$$e5756658.runUnchecked(<generated>) ~[spring-core-4.1.6.RELEASE.jar!/:2.0.0]

at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:159) [launchpad-pipeline-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:130) [launchpad-pipeline-2.0.0.jar!/:2.0.0]

at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) [guava-retrying-1.0.6.jar!/:na]

at com.github.rholder.retry.Retryer.call(Retryer.java:110) [guava-retrying-1.0.6.jar!/:na]

at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:99) [launchpad-pipeline-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:125) [launchpad-pipeline-database-2.0.0.jar!/:2.0.0]

at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) [launchpad-common-2.0.0.jar!/:2.0.0]

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_73]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_73]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_73]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_73]

at java.lang.Thread.run(Thread.java:745) [na:1.8.0_73]

Suppressed: com.cloudera.launchpad.pluggable.common.ExceptionConditions$DetailHolderException: Exception details:

  key: null

    PluginExceptionCondition{type=ERROR, message='400 Bad Request

{

  "code" : 400,

  "errors" : [ {

    "domain" : "global",

    "message" : "Invalid value for field 'resource.networkInterfaces[0].subnetwork': ''.  Subnetwork was not specified for a custom network.",

    "reason" : "invalid"

  } ],

  "message" : "Invalid value for field 'resource.networkInterfaces[0].subnetwork': ''.  Subnetwork was not specified for a custom network."

}'}

[2016-03-10 01:40:27] ERROR [pipeline-thread-3] - c.c.l.pipeline.util.PipelineRunner: (Suppressed)

com.cloudera.launchpad.pluggable.common.ExceptionConditions$DetailHolderException: Exception details:

  key: null

    PluginExceptionCondition{type=ERROR, message='400 Bad Request

{

  "code" : 400,

  "errors" : [ {

    "domain" : "global",

    "message" : "Invalid value for field 'resource.networkInterfaces[0].subnetwork': ''.  Subnetwork was not specified for a custom network.",

    "reason" : "invalid"

  } ],

  "message" : "Invalid value for field 'resource.networkInterfaces[0].subnetwork': ''.  Subnetwork was not specified for a custom network."

}'}

Thank you in advance.

[#2 · Accepted solution]

Per the Google Cloud Platform documentation on Using Subnetworks, there are two kinds of networking in Google Cloud Platform, the original Legacy (non-subnet) Networks, and the newer Subnet Networks. From the description in your post, I assume you are using Subnet Networks. The Google plugin for Cloudera Director currently supports only Legacy (non-subnet) Networks. I have filed an enhancement request issue (#118) against the plugin, and made sure developers at Cloudera and Google are aware of the issue.

[#3]

I saw that and I was hoping I read that wrong.
I'm available for beta testing if that helps move it along.

Thank you for the quick reply!

[#4]

I recommend keeping an eye on that issue in the plugin repository. If a fix appears there, you should be able to try out a local build of the plugin in your Cloudera Director installation before we ship an updated version of the plugin in a future Cloudera Director release. I will try to comment here if I hear anything about fix versions.

[#5]

I just cloned the repo and I'm looking through the code to see if I can do it myself.

[#6]

Any news regarding GCP legacy network and subnetwork?
Last time I checked (a few months ago) it was impossible to deploy in a subnetwork.

Thanks

[#7]

Hello yarivgraf,

Good news: A community member implemented subnetwork support in the Google plugin, and the work was merged several weeks ago.

https://github.com/cloudera/director-google-plugin/pull/150

The next 6.x release of Cloudera Altus Director will come packaged with a new plugin release that includes this change. In the meantime, you can build the plugin and install it into your existing Altus Director 2.8 or 6.0 installations and it should work.

* For Director 2.8, build and install the plugin from the v1.x branch.

* For Director 6.0, build and install the plugin from the v2.0.x branch.

The plugin's README describes the build process and links to some docs on installing the plugin in Altus Director. For the latter, you basically place the plugin JAR into Altus Director's plugins directory, replacing the prior Google plugin JAR.