Member since
08-24-2015
38
Posts
10
Kudos Received
7
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1552 | 03-22-2018 04:39 PM | |
2095 | 03-22-2018 02:04 PM | |
2121 | 12-07-2017 11:55 AM | |
2647 | 11-24-2017 01:02 PM | |
2055 | 03-28-2017 01:22 PM |
06-27-2019
02:37 PM
Hi GaryS, (1) Can you verify connectivity between Director host and Cloudera Manager host (maybe try pinging Cloudera Manager from Director host). The Director server logs may also contain more information on why the connection validation failed. You can refer here to see how to access the server logs: https://www.cloudera.com/documentation/director/latest/topics/director_troubleshoot.html (2) Unfortunatly, nodes can only be added/removed through the UI, it can't be done via config files. Though in your case since Director server can't reach Cloudera Manager, it would fail regardless, since this connectivity is required to properly add nodes to a cluster.
... View more
04-19-2019
03:25 PM
This probably belongs in Altus Cloud Services QA rather than the Altus Director board. https://community.cloudera.com/t5/Cloudera-Altus-Cloud-Services-Q/qa-p/Altus Yes it's possible to create a Kerberized cluster by using a 'secured' environment. You can see refer to the documentation here for more details: https://www.cloudera.com/documentation/altus/topics/alt_environment_intro.html#secure_clusters
... View more
04-19-2019
12:18 PM
Thanks dturner, I'll request to move this to the Cloudera Manager forums. First run, and the scripts associated with it are handled by Cloudera Manager, so they may have more context on whether such hacks are needed.
... View more
04-18-2019
06:16 PM
Hi dturner, I just attempted this and wasn't able to reproduce the issue. My cluster creation which included Hue succeeded using CM+CDH 6.2 with CentOS7.6. In fact I couldn't even find the offending package (wrapt) anywhere on the Hue instance, so there's something different in our environments/installation. Maybe more information about the enviornment/configuration would help narrow down the problem (whether your using preloaded/prextracted images, AWS or Azure, any special bootstrap scripts, etc)
... View more
10-08-2018
03:50 PM
@Baris Glad you got it working. With Director version 2.4+ you actually don't need to use a bootstrap script as shown in my previous example. You can just specify the CSD URL on the conf file and Director will automatically download and place it in ' /opt/cloudera/csd/'. See the documentation here: https://www.cloudera.com/documentation/director/latest/topics/director_non-cdh_products_custom_descriptors.html#concept_e2y_dpj_4z
... View more
08-03-2018
12:29 AM
1 Kudo
Glad you got it working. Just to note something like this did work for me for hvm instance types: hvm=c5.large,\ c5.xlarge,\ c5.2xlarge,\ c5d.2xlarge There should be a ec2.virtualizationmappings.properties.example file you can look at, modify as an example.
... View more
08-02-2018
11:58 PM
Hmm, so etc/ec2.virtualizationmappings.properties is the default location to specify the virtualization mapping. You can use etc/ec2.customvirtualizationmappings.properties as shown in your example but then you would have to specify this custom location in etc/aws-plugin.conf (as mentioned in the docs). Can you double check to see if that's the issue.
... View more
07-31-2018
06:54 PM
These new instance types aren't in Director yet. For now however, you should be able to update Director's AWS plugin to include them. You can refer to the documentation for the steps: https://www.cloudera.com/documentation/director/latest/topics/director_aws_new_instance_types.html
... View more
07-05-2018
03:59 PM
1 Kudo
As mentioned by dhan, you can ssh into a cluster instance and run the following. sudo yum --version If that command succeds (returns exit code 0), Director should be able to identify the package manager.
... View more
07-05-2018
10:57 AM
You can find Cloudera Management Service properties here: https://www.cloudera.com/documentation/enterprise/properties/5-15-x/topics/cm_props_mgmtservice.html
... View more
03-22-2018
05:03 PM
1 Kudo
A keypair corresponding to the private key you supplied to Director needs to exist in AWS. Can you verify that the key pair exists for that region by checking the EC2 console. If not, you can refer to the docs here to import the public key.
... View more
03-22-2018
04:39 PM
1 Kudo
Director doesn't have a built in way to specify hostnames like that. You may be able to achieve something like this through the use of bootstrapScripts which lets you run a script on the instance. Something to keep in mind, Director requires forward and reverse hostname resolutions in order to work correctly, so you will probably have to mess with DNS settings as well. This thread here may be of help.
... View more
03-22-2018
02:04 PM
1 Kudo
Thanks for reporting this jmccormick. Your right, that 5.13 repo has an incorrect manifest, we're looking into fixing this. In the meantime you can try http://archive.cloudera.com/cdh5/parcels/5.13.2 which worked fine for me.
... View more
12-08-2017
04:27 PM
That conf looks right to me. Though the service should be SPARK2_ON_YARN instead of SPARK2-ON_YARN. Also can you verify that you hava Java 8 installed which is a requirement for Spark 2, see here for additional details.
... View more
12-08-2017
04:21 PM
Any helpful information in cloudera-director-server.out ?
... View more
12-07-2017
12:31 PM
It's possible to define the security group ids and subnet id at the instance template level (same place where instance type is defined). I'm not sure if there is a sample you can refer to since most Director setups place all cluster instances in the same subnet and security group. Something to keep in mind when setting up the network is that Director requires SSH access to all instances and we generally expect full connectivity among cluster and CM instances. If you want to restrict specific ports, you can refer to the Cloudera Manager docs to see which ports are needed by CM and CDH.
... View more
12-07-2017
11:55 AM
Is this reproducible? Director does cache some of the parcel repository information but it gets cleared at the beginning of bootstrap so that shouldn't be the problem. You can also try restarting Director (which also clears the cache) just to make sure.
... View more
12-07-2017
11:42 AM
Which OS are you using? Also can you check the Director server log if it's present, it should be at /var/log/cloudera-director-server/application.log
... View more
12-07-2017
11:29 AM
Also since your dealing with multiple masters, I assume your attempting to create a High Availability cluster which is a more complex setup. You can refer to this doc and this sample reference for more information on HA.
... View more
12-07-2017
11:04 AM
Are you trying to add more than 1 HDFS Balancer role? Based on the exception message it looks like that role needs to be in a group that has 1 node.
... View more
12-05-2017
04:38 PM
1. Yes you can place the EBS properties in that section of the conf file. See here on the available EBS properties. 2. The KAFKA_BROKER role should not be placed in a gateway group, there should be a Kafka GATEWAY role for this. The sample reference configuration file provided when Director is installed (can also find here) has a fairly comprehensive list of services and roles available. Although it looks like this doc needs to be updated since it's missing the gateway role for Kafka. 3. Yes, having these gateway roles should propagate the necessary configs and allow using the cli. 4. No, Director doesn't allow placing cluster services in the CM instance. 5. Yes, Director attempts to sync with Cloudera Manager every few minutes, so you should be abel to make direct changes in CM as long as the change is consistant accross the instance group. Refer here for additional details.
... View more
12-05-2017
04:22 PM
From reading the Cloudera Manager docs, it looks like these directories will be automatically created.
... View more
12-05-2017
04:20 PM
Unfortunatly at the moment there is no way to update the scripts for an existing instance. An alternate option, although a bit painful, is to create a new instance group (with the same roles and configs) with the new bootstrap scripts and then delete the old instance group. However this will not work for the post create scripts since these are not specified at the instance level.
... View more
11-24-2017
01:02 PM
1. No, Cloudera Director can't point to existing instances, the EC2 instances must be provisioned by Director. 2. Based on my answer to (1), I don't think this question still applies. In general though, since Director manages provisioning and preparing the instances as well as installing Cloudera Manager and CDH, you'll likely run into much less compatibility issues using Director than if you were doing any of these steps manually. 3. Director can install Streamset via CSD. You can refer to this post to see an example using a bootstrap script. Note that as of Director 2.5 it's actually easier now to specify CSDs and doesn't require a bootstrap script, this is documented here.
... View more
10-26-2017
10:28 AM
What version of Cloudera Manager are you on? Director should be picking up both service and role configurations as long as you are using Cloudera Manager 5.12 or higher. Also note that Director syncs up with CM every 5 minutes so you should ensure that you've waited that long to verify that the config changes were detected.
... View more
10-26-2017
10:00 AM
Unfortuantly, this is a known limitation of Hocon, see https://github.com/lightbend/config/issues/122. Glancing at the comments, it doesn't look like there are any workarounds to get this type of substitution working.
... View more
10-26-2017
09:57 AM
include file(...) should work but I believe it requires the full file path. The current latest version of Director (Director 2.6) is on typesafe config version 1.2 which doens't have support for required. You can refer to the following spec to see what's supported in this verison: https://github.com/lightbend/config/blob/v1.2.0/HOCON.md
... View more
09-15-2017
12:29 PM
1 Kudo
You may want to check whether CM is up and running properly. It's possible that CM is out of memory which can happen when the CM instance size is too small.
... View more
07-31-2017
02:45 PM
Looking at the known issues, this isn't direclty supported in Director.
... View more
06-19-2017
12:51 PM
You may need to use SOCKS Proxy See here for additional details: https://www.cloudera.com/documentation/director/latest/topics/director_get_started_azure_socks.html
... View more