Member since
10-28-2014
82
Posts
18
Kudos Received
17
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
940 | 07-11-2019 08:49 AM | |
3731 | 08-02-2018 12:20 PM | |
4972 | 06-19-2018 02:19 PM | |
2327 | 06-19-2018 08:32 AM | |
2180 | 12-21-2017 08:19 AM |
07-12-2019
07:26 AM
please add the clusterType field to your simple conf. This field is used to determine the cluster topology (e.g., what services to run, where to locate roles). The exported conf will not include configurations that are set through CM's autoconfiguration, only those that are manually set. You may not need to set those Kudu configurations unless you want to customize them to something different than what autoconfiguration would choose.
... View more
07-11-2019
12:14 PM
You are correct in that Director will attempt to connect via each IP in the list, preferring to connect over the private IP. You should be fine if only one of the IPs is accessible. Director should cycle through the IPs until the configured timeout, default 20 minutes after instance allocation. Is the cluster failing to bootstrap? There are some natural places where we would expect connectivity to fail in a transient manner. E.g., when the VM is first allocated but doesn't yet respond to ssh, when rebooting the VM. If you are seeing failures that are preventing the cluster from bootstrapping (or even causing individual instances to fail) then we would be interested in seeing the log files to investigate.
... View more
07-11-2019
11:47 AM
The specific error you are seeing, "The configuration does not appear to be in a supported format", is because the convert-remote command doesn't think the config is a simple-setup style config. It checks this based on the inclusion of the "clusterType" field. see https://github.com/cloudera/director-scripts/blob/master/configs/aws.simple_setup.conf#L47-L51 Can you check your simple-setup conf file for this field? validate-remote can validate both simple-setup and standard conf files, so it will submit the request regardless of the presence or absence of the "clusterType" field. I'm a little surprised that the validate-remote command would work on a simple-setup conf file missing the "clusterType" field, but validation only checks for a subset of problems and would not guarantee that the cluster would bootstrap properly.
... View more
07-11-2019
08:55 AM
GaryS, The convert-remote command is used to convert simple-setup style conf files into full conf files. e.g., files like https://github.com/cloudera/director-scripts/blob/master/configs/aws.simple_setup.conf into files like https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf Can you tell us a little more about what you are trying to accomplish and about the conf file you are trying to convert?
... View more
07-11-2019
08:49 AM
dturner, We don't have any documented recommendations for the max deployments or clusters. Anecdotally, I've heard of 100+ clusters. Director does the most work when bootstrapping/updating/terminating clusters and less when just monitoring clusters. You are also running on a way larger instance than we usually use so I bet there's room for some tuning if you do run into issues (e.g., increasing memory, threadpools, etc).
... View more
07-10-2019
08:36 AM
"enableEnterpriseTrial" should have no effect on java installation. Also, the java rpm should be available in the CM yum repo. What version of Director are you using and what version of CM are you installing? I do see that https://archive.cloudera.com/cm6/6.2.0/ shows a "last modified" date of yesterday (7/9). Perhaps there was some maintenance being performed on the repo when you ran. Can you try to run again?
... View more
03-29-2019
11:12 AM
followup with a little more context and clarification. I mis-spoke when I said that " the created users are not being granted the correct role to operate on the created database." In actuality, the role provided to Altus Director is not able to create a database with an owner other than itself. The database provisioning logic attempts to create a separate role and use that as the owner for each database. SUPERUSER privileges are required to create a database with a different owner . Google cloud sql does not support giving SUPERUSER privileges (see https://cloud.google.com/sql/docs/postgres/features under "Unsupported Features"). External Database Templates should work correctly if you create your own Postgres server and provide Altus Director with a SUPERUSER role.
... View more
03-27-2019
12:02 PM
1 Kudo
cgiardina, Altus Director is not able to manage (i.e., create/delete) cloudsql postgres databases due to a permissions issue. As you can see by the error messages, the created users are not being granted the correct role to operate on the created database. You can still use postgres databases that you create and manage yourself. See the documentation and reference conf file for information on External Databases, not External Database Templates. https://www.cloudera.com/documentation/director/latest/topics/director_external_db_using.html https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf#L629-L653 https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf#L993-L1034
... View more
09-20-2018
12:07 PM
I think that this log line highlights the error. PluginExceptionCondition{type=ERROR, exceptionInfo={message=Encountered AWS exception, awsErrorCode=InvalidParameterValue, awsErrorMessage=Duplicate tag key 'Name' specified.}} Are you trying to add a "Name" tag to your instances? Cloudera Altus Director uses the "Name" tag, but can be re-configured to use a different tag instead. See https://www.cloudera.com/documentation/director/latest/topics/director_aws_custom_tag_names.html for instructions on how to do so.
... View more
08-02-2018
12:20 PM
1 Kudo
iasindev, The symptoms you describe are due to a bug in the 2.8.0 standalone client. The standalone client commands are deprecated in 2.8.0 and unfortunately suffer from this bug. In preference order, workarounds are: 1. Use the bootstrap-remote command with a Cloudera Director server. This is the preferred configuration. or 2. Unset the segment analytics writeKey property for the cli by adding segment.analytics.writeKey: to /etc/cloudera-director-client/application.properties or 3. Use the 2.7.1 standalone client. It does not suffer from this bug. Good luck! David
... View more
07-20-2018
08:28 AM
1 Kudo
ppiper, The port forwarding errors can generally be ignored. Cloudera Director is polling the ssh port on the instance to wait until the instance starts responding. The failure is likely indicated by this in the second set of logs you included PluginExceptionCondition{type=ERROR, exceptionInfo={message=403 Forbidden
{
"code" : 403,
"errors" : [ {
"domain" : "usageLimits",
"message" : "Quota 'CPUS' exceeded. Limit: 8.0 in region us-east1.",
"reason" : "quotaExceeded"
} ],
"message" : "Quota 'CPUS' exceeded. Limit: 8.0 in region us-east1."
}}} It looks like you exceeded your quota in Google Cloud. David
... View more
07-20-2018
08:24 AM
robinwood, I'm afraid that there's not enough information in your log snippet for me to tell what failed. Can you supply a larger portion of the logs preceeding this error? Log lines marked with [ p-dfc30dc4e9a3-DefaultBootstrapDeploymentJob] are for this thread that experienced the error. Also look for some [io-thread-?] log lines that are the output from the script execution. Those may give us some more clues as to why the error occurred.
... View more
07-17-2018
08:17 AM
Cloudera Director can install Kudu using conf files and the CLI bootstrap-remote command. In addition to the linked documentation above, also see the aws.reference.conf. The cluster defined in the reference conf file includes Kudu. If you want to install Kudu on a Director-managed cluster that already exists, you need to follow the Cloudera Manager's documentation to do so. Director will "refresh" it's models of the cluster to include any added services or configurations. See Director's documentation on cluster management for a description of which tasks should be performed in Director vs. CM.
... View more
06-19-2018
02:19 PM
1 Kudo
sg321, I've followed the instructions you linked through cluster bootstrap and successfully created a cluster using ami-2051294a in us-east-1. I did have to make 1 modification to get the spark2 csd to match the spark2 parcel repo. You can do this by either bumping the csd up to http://archive.cloudera.com/spark2/csd/SPARK2_ON_YARN-2.2.0.cloudera2.jar or pinning the parcel repo to http://archive.cloudera.com/spark2/parcels/2.2.0.cloudera2/ Is this reliably/repeatably failing first run for you?
... View more
06-19-2018
08:32 AM
1 Kudo
Tomas79, Please refer to the CM and CDH documentation. Use the "API Name" https://www.cloudera.com/documentation/enterprise/properties/5-15-x/topics/cm_props_cdh5150.html Note that "Service-wide" configurations are specified at the cluster level in the serviceConfigs block while the role-specific configurations are specified per instance group in the roleTypeConfigs blocks. David
... View more
06-18-2018
08:57 AM
sg321, Please look at this article. http://community.cloudera.com/t5/CDH-Manual-Installation/How-to-resolve-quot-Permission-denied-quot-errors-in-CDH/ta-p/36141 I find it strange that you are getting this error on a clean cluster bootstrap. Can you share your conf file? Please redact any secrets (e.g., aws keys). What version of Director are you using? David
... View more
06-12-2018
08:07 AM
1 Kudo
sg321, You do need to look in CM to find the root cause of the First Run failure. The CM UI has a tab "All Recent Commands". https://www.cloudera.com/documentation/enterprise/5-12-x/topics/cm_dg_view_running_recent_commands.html First Run is composed of a series of steps. You can determine which step failed through the UI. You can also find this information in the scm-server-log. Information about the step that failed should appear somewhere above the scm-server-log message that you posted. David
... View more
04-09-2018
08:19 AM
1 Kudo
alexmc6, Based on the terminology used, I'm assuming that you are referring to AWS. Please refer to the Cloudera Enterprise Reference Architecture for AWS Deployments. "Appendix A: Spanning AWS Availability Zones" will provide the guidance you are looking for. In Director, each Environment corresponds to a single region, but you can (indirectly) select the AZ used for each node by setting the subnet in the InstanceTemplate. Refer to the aws.reference.conf for an example of where you can override the subnetId and set the rackId. Note that, since a single InstanceTemplate is used for each instance group, you will need to make separate instance groups for each AZ. David
... View more
02-28-2018
02:08 PM
Sentara, Thank you very much for posting your approach here. I'm sure that it will help future community users. David
... View more
02-22-2018
07:48 AM
Sentara, You are correct that your DNS must be able to resolve the ip-X-X-X-X@ourcompany.com names or whatever names you want your hosts to use. Definitely review AWS's documentation https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-dns.html Here's some suggestions: 1. You can run a bootstrap script to your InstanceTemplates to add your hostname to DNS using dhclient or NetworkManager if your DNS server is configured to allow updates from your VPC instances. We have published some scripts to help people using Cloudera Director in Azure. You can refer to and adapt these scripts to your network. See https://github.com/cloudera/director-scripts/tree/master/azure-dns-scripts 2. You can brute force the problem by pre-registering all potential hostnames in your VPC or subnet with your DNS server. The subnets are configured to use IP addresses in a specified range and the hostnames are derived from those IP addresses in a known, systematic way. Thus, you can produce the expected hostnames for each IP address in your subnet and add those to your zone files. 3. You can investigate using a bootstrap script to obtain a company-specific hostname that is resolvable in your company DNS and configure your instance to use that hostname instead of the ip-X-X-X-X hostname. Please report back any progress or solutions that work for you to share the knowledge with the rest of the community. If anyone else in the community has any suggestions, please chime in! Thanks, David
... View more
01-26-2018
11:56 AM
1 Kudo
Always use the configuration API name with Cloudera Director. See https://www.cloudera.com/documentation/enterprise/properties/5-14-x/topics/cm_props_cdh5140_hue.html#concept_5.14.x_loadbalancer_props__section_ports_and_addresses_props for the API names. This is a Role config (not service-wide) so the config must be placed in each instance group that contains a hue load balancer role. e.g., cluster { configs { # service-wide configs are placed at the cluster level }
instance-group-name {
roles {
HUE: [HUE_LOAD_BALANCER]
}
configs { # role configs are placed in the instance groups
HUE {
HUE_LOAD_BALANCER {
listen: 8889
}
}
}
}
... View more
01-26-2018
11:44 AM
Rather than restoring from a snapshot, could you stop and start the instance instead? You wouldn't need to reconfigure anything in Director using this method. While stopped, you could re-attach new EBS volumes created from your snapshot. See https://www.cloudera.com/documentation/director/latest/topics/director_server_startstop.html If this workflow doesn't work for you, could you provide more information about your use case?
... View more
12-21-2017
08:19 AM
dturner, The clusters will be independent. There is one minor caveat. Director finds the instances in AWS based on their "Cloudera-Director-Id" tag. Cloudera Director uses UUIDs for these so they will be unique across all instances. However, Cloudera Director accepts any String for this ID if you are creating your cluster through the API (i.e., not through Cloudera Director's UI or CLI). In that case, it is incumbent upon you to ensure you don't have any ID conflicts. Here's an example of how you could get into trouble. If you GET a ClusterTemplate through the API, change the "name", and POST it back to create a new Cluster. Director may find the wrong instances in AWS. To avoid this, you must also change all the "id" fields for all of the "virtualInstances".
... View more
12-21-2017
08:02 AM
TestUser, Cloudera Director 2.6.x is configured with repositories for CM/CDH 5.13 by default. See the documentation for how to configure a non-default repository. You can look for the releant section in the instructions for any of the cloud providers. https://www.cloudera.com/documentation/director/latest/topics/director_deployment_requirements.html#concept_sm5_hxt_lt In addition to setting the CDH product version, you need to specify the parcel repository. Please see the links to the reference conf file in my previous reply. There is a field you can fill out if you are creating a cluster through the UI. At the bottom of that page, you can open up the "Configurations" and check "Override default parcel repositories". You can specify http://archive.cloudera.com/cdh5/parcels/5.12/ and any other parcel repos you need. David
... View more
12-20-2017
11:59 AM
TestUser, Cloudera Manager is compatible with older versions of CDH. https://www.cloudera.com/documentation/enterprise/release-notes/topics/rn_consolidated_pcm.html#cm_cdh_compatibility Do you have the exact error? Does Cloudera Director throw the error or does Cloudera Manager? You specify the CDH version to Director by specifying both the product version and the parcels repository. https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf#L794 https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf#L804 A mismatch between these fields would cause an issue while bootstrapping. David
... View more
12-19-2017
10:09 AM
Andy, The best place to look is the agent install logs and the agent logs. /tmp/scm_prepare_node.<Unique ID> /var/log/cloudera-scm-agent You should also check that your security group allows full access from other cluster instances (e.g., from other instances in the same security group). It also looks like you are using custom DNS, but I still see the .ec2.internal addresses in the HostEndpoint list. If you've set up your DHCP Option Set to point to your own DNS server then you should disable DNS Hostnames and/or DNS Resolution on your VPC. David
... View more
11-09-2017
01:34 PM
Tauqeer, Cloudera Director's API will provide what you need to add instances and replace data nodes. Master migration is a little trickier. Cloudera Director can help with creating instances for the migration, but you will have to perform the migration in Cloudera Manager. See https://www.cloudera.com/documentation/director/latest/topics/director_create_ha_clusters.html#migrate_hdfs_master_roles for information on how to use Cloudera Director when migrating masters. These tasks will mainly use Cloudera Director's "update" API. See https://github.com/cloudera/director-sdk/blob/master/java-client/src/main/java/com/cloudera/director/client/latest/api/ClustersApi.java#L549 or https://github.com/cloudera/director-sdk/blob/master/python-client/cloudera/director/latest/ClustersApi.py#L625 There are examples in both java and python to get you started. https://github.com/cloudera/director-sdk
... View more
11-09-2017
01:24 PM
David, Cloudera Director 2.5 changed the way it tags instances to include the tags on the runInstances request. This feature has not been added to the China region. I'm afraid that you will have to stay on 2.4.x until Amazon updates that region to support this features. A proxy server will work. Which yum packages are timing out? Here are other alternatives. You could create an ami with the packages pre-installed. https://github.com/cloudera/director-scripts/tree/master/faster-bootstrap You could host the parcels and packages. If you do this, then you will need to configure the custom repos when you create a Deployment or Cluster. https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_create_local_package_repo.html https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_create_local_parcel_repo.html Good Luck! David
... View more
09-28-2017
10:13 AM
Sankaranarayanan, Cloudera Director does not support proxy exclusions. This like likey most easily configured on your proxy server. The proxy server should have a configuration to not proxy local addresses. David
... View more
09-27-2017
12:25 PM
Shailesh, Thanks for bringing this up. We are aware of the problem and are actively working on a fix for the tck. In the meantime, you can test your plugin by replacing the plugin that was installed by Cloudera Director with your own version. David
... View more