please add the clusterType field to your simple conf. This field is used to determine the cluster topology (e.g., what services to run, where to locate roles).   The exported conf will not include configurations that are set through CM's autoconfiguration, only those that are manually set. You may not need to set those Kudu configurations unless you want to customize them to something different than what autoconfiguration would choose. ... View more

The specific error you are seeing,  "The configuration does not appear to be in a supported format", is because the convert-remote command doesn't think the config is a simple-setup style config. It checks this based on the inclusion of the "clusterType" field. see https://github.com/cloudera/director-scripts/blob/master/configs/aws.simple_setup.conf#L47-L51   Can you check your simple-setup conf file for this field?    validate-remote can validate both simple-setup and standard conf files, so it will submit the request regardless of the presence or absence of the "clusterType" field. I'm a little surprised that the validate-remote command would work on a simple-setup conf file missing the "clusterType" field, but validation only checks for a subset of problems and would not guarantee that the cluster would bootstrap properly. ... View more

GaryS,   The convert-remote command is used to convert simple-setup style conf files into full conf files. e.g., files like https://github.com/cloudera/director-scripts/blob/master/configs/aws.simple_setup.conf into files like https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf   Can you tell us a little more about what you are trying to accomplish and about the conf file you are trying to convert? ... View more

"enableEnterpriseTrial" should have no effect on java installation. Also, the java rpm should be available in the CM yum repo. What version of Director are you using and what version of CM are you installing?   I do see that  https://archive.cloudera.com/cm6/6.2.0/ shows a "last modified" date of yesterday (7/9). Perhaps there was some maintenance being performed on the repo when you ran. Can you try to run again? ... View more

followup with a little more context and clarification.   I mis-spoke when I said that " the created users are not being granted the correct role to operate on the created database." In actuality, the role provided to Altus Director is not able to create a database with an owner other than itself. The database provisioning logic attempts to create a separate role and use that as the owner for each database. SUPERUSER privileges are required to create a database with a different owner . Google cloud sql does not support giving SUPERUSER privileges (see https://cloud.google.com/sql/docs/postgres/features under "Unsupported Features").   External Database Templates should work correctly if you create your own Postgres server and provide Altus Director with a SUPERUSER role. ... View more

cgiardina,   Altus Director is not able to manage (i.e., create/delete) cloudsql postgres databases due to a permissions issue. As you can see by the error messages, the created users are not being granted the correct role to operate on the created database.   You can still use postgres databases that you create and manage yourself. See the documentation and reference conf file for information on External Databases, not External Database Templates. https://www.cloudera.com/documentation/director/latest/topics/director_external_db_using.html https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf#L629-L653 https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf#L993-L1034   ... View more

ppiper,   The port forwarding errors can generally be ignored. Cloudera Director is polling the ssh port on the instance to wait until the instance starts responding.   The failure is likely indicated by this in the second set of logs you included PluginExceptionCondition{type=ERROR, exceptionInfo={message=403 Forbidden { "code" : 403, "errors" : [ { "domain" : "usageLimits", "message" : "Quota 'CPUS' exceeded. Limit: 8.0 in region us-east1.", "reason" : "quotaExceeded" } ], "message" : "Quota 'CPUS' exceeded. Limit: 8.0 in region us-east1." }}} It looks like you exceeded your quota in Google Cloud.   David ... View more

robinwood,   I'm afraid that there's not enough information in your log snippet for me to tell what failed. Can you supply a larger portion of the logs preceeding this error?   Log lines marked with [ p-dfc30dc4e9a3-DefaultBootstrapDeploymentJob] are for this thread that experienced the error. Also look for some [io-thread-?] log lines that are the output from the script execution. Those may give us some more clues as to why the error occurred.   ... View more

Cloudera Director can install Kudu using conf files and the CLI bootstrap-remote command.  In addition to the linked documentation above, also see the aws.reference.conf. The cluster defined in the reference conf file includes Kudu.   If you want to install Kudu on a Director-managed cluster that already exists, you need to follow the Cloudera Manager's documentation to do so. Director will "refresh" it's models of the cluster to include any added services or configurations. See Director's documentation on  cluster management for a description of which tasks should be performed in Director vs. CM. ... View more

alexmc6,   Based on the terminology used, I'm assuming that you are referring to AWS. Please refer to the Cloudera Enterprise Reference Architecture for AWS Deployments. "Appendix A: Spanning AWS Availability Zones" will provide the guidance you are looking for.   In Director, each Environment corresponds to a single region, but you can (indirectly) select the AZ used for each node by setting the subnet in the InstanceTemplate. Refer to the aws.reference.conf for an example of where you can override the subnetId and set the rackId. Note that, since a single InstanceTemplate is used for each instance group, you will need to make separate instance groups for each AZ.   David ... View more

Sentara,   Thank you very much for posting your approach here. I'm sure that it will help future community users.   David   ... View more

Sentara,   You are correct that your DNS must be able to resolve the ip-X-X-X-X@ourcompany.com names or whatever names you want your hosts to use. Definitely review AWS's documentation https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-dns.html   Here's some suggestions:   1. You can run a bootstrap script to your InstanceTemplates to add your hostname to DNS using dhclient or NetworkManager if your DNS server is configured to allow updates from your VPC instances. We have published some scripts to help people using Cloudera Director in Azure. You can refer to and adapt these scripts to your network. See https://github.com/cloudera/director-scripts/tree/master/azure-dns-scripts   2. You can brute force the problem by pre-registering all potential hostnames in your VPC or subnet with your DNS server. The subnets are configured to use IP addresses in a specified range and the hostnames are derived from those IP addresses in a known, systematic way. Thus, you can produce the expected hostnames for each IP address in your subnet and add those to your zone files.   3. You can investigate using a bootstrap script to obtain a company-specific hostname that is resolvable in your company DNS and configure your instance to use that hostname instead of the ip-X-X-X-X hostname.   Please report back any progress or solutions that work for you to share the knowledge with the rest of the community.   If anyone else in the community has any suggestions, please chime in!     Thanks, David ... View more

Always use the configuration API name with Cloudera Director. See https://www.cloudera.com/documentation/enterprise/properties/5-14-x/topics/cm_props_cdh5140_hue.html#concept_5.14.x_loadbalancer_props__section_ports_and_addresses_props for the API names.   This is a Role config (not service-wide) so the config must be placed in each instance group that contains a hue load balancer role.   e.g.,  cluster { configs { # service-wide configs are placed at the cluster level } instance-group-name { roles { HUE: [HUE_LOAD_BALANCER] } configs { # role configs are placed in the instance groups HUE { HUE_LOAD_BALANCER { listen: 8889 } } } }   ... View more

Rather than restoring from a snapshot, could you stop and start the instance instead? You wouldn't need to reconfigure anything in Director using this method. While stopped, you could re-attach new EBS volumes created from your snapshot. See https://www.cloudera.com/documentation/director/latest/topics/director_server_startstop.html   If this workflow doesn't work for you, could you provide more information about your use case?     ... View more

TestUser,   Cloudera Director 2.6.x is configured with repositories for CM/CDH 5.13 by default. See the documentation for how to configure a non-default repository. You can look for the releant section in the instructions for any of the cloud providers. https://www.cloudera.com/documentation/director/latest/topics/director_deployment_requirements.html#concept_sm5_hxt_lt   In addition to setting the CDH product version, you need to specify the parcel repository. Please see the links to the reference conf file in my previous reply. There is a field you can fill out if you are creating a cluster through the UI. At the bottom of that page, you can open up the "Configurations" and check "Override default parcel repositories". You can specify http://archive.cloudera.com/cdh5/parcels/5.12/ and any other parcel repos you need.   David     ... View more

TestUser,   Cloudera Manager is compatible with older versions of CDH. https://www.cloudera.com/documentation/enterprise/release-notes/topics/rn_consolidated_pcm.html#cm_cdh_compatibility   Do you have the exact error? Does Cloudera Director throw the error or does Cloudera Manager?   You specify the CDH version to Director by specifying both the product version and the parcels repository. https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf#L794 https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf#L804 A mismatch between these fields would cause an issue while bootstrapping.   David ... View more

Armen,   Are there any errors in the Cloudera Director log file? I would not expect that terminating a Cloudera Manager would prevent Cloudera Director from starting. The failure to start should have some other cause.   Once started, you can "Terminate" the deployment in Cloudera Director's UI. This will stop the ping attempts.     David ... View more