Hi ztoth,   When Cloudera Manager returns a 400 error, often there are clues for what went wrong in Cloudera Manager's own logs (/var/log/cloudera-scm-server). If there's not much to go on there, then it's worth trying again, but configuring Cloudera Manager to emit debug information related to API calls. Pass this configuration property for Cloudera Manager in the deployment template:   enable_api_debug: true   There's an example for this in our reference configuration file for AWS:   https://github.com/cloudera/director-scripts/blob/master/configs/aws.reference.conf   This may or may not be the issue, but you should start a cluster with at least three workers, because each of them hosts an HDFS datanode, and you need at least three of those to meet the default HDFS replication factor. ... View more

Hi ztoth,   The API doesn't support referencing an existing instance template when creating a new deployment through it. The entire instance template for the manager instance must be spelled out. To make this easier, you can GET the template for an existing deployment (not the deployment itself, but its template) that uses the desired instance template, and then copy out the details from the returned JSON.   Going further, you can edit the JSON from that existing deployment template into what you POST for the new one, essentially rolling your own clone operation. If you do this, be sure to update the rest of the template to make it properly independent of the original: change the name of the deployment, change the ID for the manager virtual instance, update any bootstrap scripts, fill in the license and billing ID if used, and update database names and credentials for any pre-existing external databases.   If you're also looking at reusing an external database server, once it's created it can be referenced inside external database templates and (existing) external databases in new deployment templates by its name, and it will be used as expected. ... View more

Hi Garren,   You can configure Director to work with a new EC2 instance type without waiting for a new Director release. We implemented the AWS plugin this way because new instance types appear all the time, and you shouldn't have to wait for us to be able to use them.   https://www.cloudera.com/documentation/director/latest/topics/director_aws_new_instance_types.html   Even once you configure for a new instance type, though, it's still possible that the AMI you chose is not available for it. For example, my favorite in us-east-1, ami-6d1c2007, can't use an i3 instance type when I try to launch one from the AWS console: "The instance configuration for this AWS Marketplace product is not supported." ... View more

Hello zhuw.bigdata,   This is a known issue with Director's automated generation of passwords for MySQL databases. There is a fix for it included with Director 2.3, which is due to be released soon. In the meantime, you can try adjusting the password validation level in MySQL 5.7 to LOW, instead of the default MEDIUM, and Director's passwords should be accepted.   Director uses ordinary random UUIDs as passwords, and they fail the MEDIUM level of MySQL password validation because they lack an uppercase letter (and may, in very rare cases, lack either a lowercase letter or a number). ... View more

Hello there,   Director uses Cloudera Manager's API under the hood to configure services. The selection of parameters supported for HBase can be found here.   http://www.cloudera.com/documentation/enterprise/properties/5-8-x/topics/cm_props_cdh580_hbase.html   You can include the parameters you need in service or role configurations specified through Director, either in the UI or in a configuration file. Use the name in the "API" column of the page linked above to name each property. For example, to specify the region server message interval, include the "hbase_regionserver_msginterval" configuration property in the role configuration for region servers. A configuration file would specify it like this, perhaps in an "workers" instance group.   roles {   ...   HBASE: [REGIONSERVER]   ... } configs {   HBASE {     REGIONSERVER {       hbase_regionserver_msginterval: 10     }   } }   Take a look at the sample configuration files shipped with Director for a complete example.   https://www.cloudera.com/documentation/director/latest/topics/director_cluster_config.html   General documentation about setting these kinds of configuration properties is here:   https://www.cloudera.com/documentation/director/latest/topics/director_setting_cm_configs.html ... View more

Hi kerjo,   We're actively looking into the issue of ntpd on CentOS 7 and RHEL 7. What we understand is that for those OS versions, chronyd has replaced ntpd as the preferred time synchronization daemon. It's likely that the AMI(s) you are using have chronyd installed and that is causing a conflict.   If chronyd is running on the cluster nodes despite how ntpd behaves, then you should be in the clear. You won't need ntpd running.   Otherwise, until we adjust our instance normalization logic, you can define an instance template bootstrap script for the cluster nodes that installs and starts chronyd, and makes it enabled. I think that even though Director's normalization logic will subsequently still attempt to work with ntpd, chronyd will still run and be unaffected. You can find information about bootstrap scripts in the sample configuration files, documented here: www.cloudera.com/documentation/director/latest/topics/director_cluster_config.html.   yum install chronyd systemctl enable chronyd.service systemctl start chronyd.service Hopefully that will get things going for you. If you still end up with neither ntpd or chronyd running on the cluster nodes, or if Director reports problems working with ntpd due to the conflict, let us know and we can investigate some more.   Bill ... View more