Proper way to add a new cluster to existing Cloudera Manager using a config file?

[#1]

What is the right way to re-use (add to) an existing Cloudera Manager using a config file?
We are trying aws.ha.reference.conf with this setting and CM paragraph:

deploymentName: My_Test" Deployment"
environmentName: myCM" Environment"
// ...
cloudera-manager {
username: admin
password: admin
}

Run like this:

cloudera-director bootstrap-remote aws.ha.reference.conf --lp.remote.username=admin --lp.remote.password=xxxxxxxx --lp.remote.hostAndPort=localhost:7189

The result is:

Cloudera Director 2.1.0 initializing ...
Connecting to http://localhost:7189
Current user roles: [ROLE_ADMIN, ROLE_READONLY]
Found errors in deployment configuration:
* You have to either specify a virtual instance configuration or a Cloudera Manager URL for a new deployment

-----------
How can I specify an existing instance? I am referring to a CM that was created by Cloudera Director.

[#2]

Based on the error message, I don't think Cloudera Director is finding your existing deployment, and is trying to create a new one. Although I believe HOCON is forgiving about the use of adjacent unquoted and quoted strings, can you please move the open quotes on your deploymentName and environmentName to before MyTest and myCM? Also, are you certain that those are the correct names of the deployment and environment in which you deployed your existing cluster(s)? You can verify that in the Cloudera Director UI.

[#3]

We tried it with and without " Deployment" and " Environment" appended, but alway get the same error. We have successfully added more than one cluster to the particular CM. 

The tail end of the application.log is below, if that helps. 

Alternatively, how can we specify "Cloudera Manager URL"?

[2016-09-15 02:11:34] INFO  [main] - c.c.l.m.PrivateKeySshCredentialsValidator: Validating SSH credentials for ec2-user
[2016-09-15 02:11:34] ERROR [main] - c.c.l.commands.ValidateCommand: Failed to parse deployment configuration
java.lang.IllegalArgumentException: You have to either specify a virtual instance configuration or a Cloudera Manager URL for a new deployment
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:93) ~[guava-15.0.jar!/:na]
        at com.cloudera.launchpad.model.deployment.DeploymentTemplate.<init>(DeploymentTemplate.java:154) ~[launchpad-model-2.1.0.jar!/:2.1.0]
        at com.cloudera.launchpad.model.deployment.DeploymentTemplateBuilder.build(DeploymentTemplateBuilder.java:474) ~[launchpad-model-2.1.0.jar!/:2.1.0]
        at com.cloudera.launchpad.templates.ConfigToDeploymentTemplate.apply(ConfigToDeploymentTemplate.java:183) ~[launchpad-templates-2.1.0.jar!/:2.1.0]
        at com.cloudera.launchpad.commands.ValidateCommand.run(ValidateCommand.java:136) ~[launchpad-cli-2.1.0.jar!/:2.1.0]
        at com.cloudera.launchpad.commands.BootstrapRemoteCommand.run(BootstrapRemoteCommand.java:74) [launchpad-cli-2.1.0.jar!/:2.1.0]
        at com.cloudera.launchpad.commands.RemoteCommand.run(RemoteCommand.java:198) [launchpad-cli-2.1.0.jar!/:2.1.0]
        at com.cloudera.launchpad.Application.run(Application.java:140) [launchpad-cli-2.1.0.jar!/:2.1.0]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:806) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:790) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
        at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:777) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:308) [spring-boot-1.3.2.RELEASE.jar!/:1.3.2.RELEASE]
        at com.cloudera.launchpad.Application.start(Application.java:97) [launchpad-cli-2.1.0.jar!/:2.1.0]
        at com.cloudera.launchpad.Application.main(Application.java:47) [launchpad-cli-2.1.0.jar!/:2.1.0]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_72]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_72]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_72]
        at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_72]
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:54) [launchpad-cli-2.1.0.jar!/:2.1.0]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
[2016-09-15 02:11:34] INFO  [Thread-9] - o.s.c.a.AnnotationConfionConfigApplicationContext@4f04b01a: startup date [Thu Sep 15 02:11:15 UTC 2016]; root of context hierarchy

[#4 · Accepted solution]

Give that Cloudera Manager was provisioned using Director, I don't think we want to go down the route of supplying a CM URL.

What @jadair was saying is that instead of

deploymentName: My_Test" Deployment"
environmentName: myCM" Environment"

try:

deploymentName: "My_Test Deployment"
environmentName: "myCM Environment"

Also, the question about verifying the names of the deployment/environment are still relevant here. The name of the deployment and environment aren't necessarily suffixed with Deployment and Environment -- they can be named arbitrarily. By default, however, the Director client suffixes deployment and environment names with those strings.

Finally, you may have to specify an image for this CM, even though it won't get used. This is due to a limitation on how we do our validation. For example:

deploymentName: "My_Test Deployment"
environmentName: "myCM Environment"
...
instances {
    cm-image {
        type: m3.xlarge
        image: ami-6283a827
    }
}
....
cloudera-manager {
    instance: ${instances.cm-image} {
        tags {
            application: "Cloudera Manager 5"
        }
    }
    ....
    username: "my_username"
    password: "my_password"
}

Let me know if any of this helps.

[#5]

That helped. It works now.

We had no reason to expect that the instance and instances clauses were necessary for a CM that already exists and was generated by Cloudera Director.