Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Update to existing cluster using director client.

Update to existing cluster using director client.

Explorer

So far no success doing update to exising cluster, have read documentation many times for update still no luck.

I am using cloudera director client only.

 

Step 1 :
Cloned old conf file aws.reference.conf with new name aws.reference-update.conf.
Command : cloudera-director update aws.reference-update.conf (increased worker count to 3)
Changes done : Only increased number of workers, entire conf as is with new name.

 

workers {

count: 3

.....

}


Error : com.cloudera.launchpad.api.common.DeploymentsResource$UnsupportedDeploymentUpdateException: Only username and password changes are supported
at com.cloudera.launchpad.api.common.DeploymentsResource.verifyCanUpdateDeploymentTemplate(Deploym entsResource.java:369) ~[launchpad-api-common-2.0.0.jar!/:2.0.0]
at com.cloudera.launchpad.api.common.DeploymentsResource.update(DeploymentsResource.java:341) ~[la unchpad-api-common-2.0.0.jar!/:2.0.0]
at com.cloudera.launchpad.api.common.DeploymentsResource.update(DeploymentsResource.java:314) ~[la unchpad-api-common-2.0.0.jar!/:2.0.0]
at com.cloudera.launchpad.commands.UpdateCommand.run(UpdateCommand.java:111) ~[launchpad-cli-2.0.0 .jar!/:2.0.0]
at com.cloudera.launchpad.Application.run(Application.java:140) [launchpad-cli-2.0.0.jar!/:2.0.0]
at org.springframework.boot.SpringApplication.runCommandLineRunners(SpringApplication.java:672) [s pring-boot-1.2.4.RELEASE.jar!/:1.2.4.RELEASE]
at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:690) [spring-boo t-1.2.4.RELEASE.jar!/:1.2.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:321) [spring-boot-1.2.4.R ELEASE.jar!/:1.2.4.RELEASE]
at com.cloudera.launchpad.Application.start(Application.java:97) [launchpad-cli-2.0.0.jar!/:2.0.0]
at com.cloudera.launchpad.Application.main(Application.java:47) [launchpad-cli-2.0.0.jar!/:2.0.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_91]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_91]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8. 0_91]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53) [launchpad-cli-2 .0.0.jar!/:2.0.0]
at

Step 2:
Removed entire sections from aws.reference-update.conf and only kept worker section with higher count for workers (3).
Command : cloudera-director update aws.reference-update.conf (worker count to 3)

Error. Updating environment state ...No environment found for this cluster


Step 3:

Used original aws.reference.conf with number of workers to 3 now.
Command : cloudera-director update aws.reference.conf (worker count to 3)

Changes : only increased number of workers.

 

Error :

 

Unexpected internal error (see logs): You have to either specify a virtual instance configuration or a Cloudera Manager URL for a new deployment

[2016-06-10 04:15:20] ERROR [main] - com.cloudera.launchpad.Application: Command execution failed
java.lang.IllegalArgumentException: You have to either specify a virtual instance configuration or a Cloudera Manager URL for a new deployment
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:93) ~[guava-15.0.jar!/:na]
at com.cloudera.launchpad.model.deployment.DeploymentTemplate.<init>(DeploymentTemplate.java:143) ~[launchpad-model-2.0.0.jar!/:2.0.0]
at com.cloudera.launchpad.model.deployment.DeploymentTemplateBuilder.build(DeploymentTemplateBuilder.java:452) ~[launchpad-model-2.0.0.jar!/:2.0.0]
at com.cloudera.launchpad.templates.ConfigToDeploymentTemplate.apply(ConfigToDeploymentTemplate.java:182) ~[launchpad-templates-2.0.0.jar!/:2.0.0]
at com.cloudera.launchpad.commands.UpdateCommand.run(UpdateCommand.java:105) ~[launchpad-cli-2.0.0.jar!/:2.0.0]
at com.cloudera.launchpad.Application.run(Application.java:140) [launchpad-cli-2.0.0.jar!/:2.0.0]
at org.springframework.boot.SpringApplication.runCommandLineRunners(SpringApplication.java:672) [spring-boot-1.2.4.RELEASE.jar!/:1.2.4.RELEASE]
at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:690) [spring-boot-1.2.4.RELEASE.jar!/:1.2.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:321) [spring-boot-1.2.4.RELEASE.jar!/:1.2.4.RELEASE]
at com.cloudera.launchpad.Application.start(Application.java:97) [launchpad-cli-2.0.0.jar!/:2.0.0]
at com.cloudera.launchpad.Application.main(Application.java:47) [launchpad-cli-2.0.0.jar!/:2.0.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_91]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_91]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53) [launchpad-cli-2.0.0.jar!/:2.0

 

 

Regards,

Kartik

2 REPLIES 2

Re: Update to existing cluster using director client.

Expert Contributor

Was your "existing cluster" created using the bootstrap command of the Cloudera Director client with your starting aws.reference.conf? Did bootstrapping succeed? Is the cluster in a usable state? If any of these are not true I would not expect the update to succeed. You may want to try bootstrapping a new cluster with the config file, changing the value of the "name" attribute of the cluster.

 

Your error in Step 1 suggests that something in the cloudera-manager section of the config file (other than username and password) changed between config file used for the original bootstrapping and the config file used for the update command. Please make a copy of your original and updated config files, replace any sensitive information like passwords, and attach them here.

 

Your error in Step 2 suggests that you removed too much from the config file. If the environment name is not specified via the "environmentName" attribute, it is computed by appending " Environment" to the cluster's "name" attribute value. You probably deleted the cluster name.

 

Your error in Step 3 suggests that the original bootstrapping failed to create the Cloudera Manager instance, which is why I asked my intro questions above.

 

Highlighted

Re: Update to existing cluster using director client.

Explorer

Existing cluster was created using "cloudera-director bootstrap aws.reference.conf" command successfully.  We are able to use cluster very well. Not able to increase number of workers.

 

I only increased count and mincount to 3. Cluster was created using 1.

 

workers {
count: 3
#
# Minimum number of instances required to set up the cluster.
# Fail and quit if minCount number of instances is not available in this cloud
# environment. Else, continue setting up the cluster.
#
minCount: 3

instance: ${instances.i2x} {

# Put all cluster nodes in a placement group for improved network performance
# Note: this only works for a limited set of instances
# http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/placement-groups.html

# placementGroup: REPLACE-ME

tags {
group: worker
}
}

 

I used update command to increase workers.