Member since
10-28-2014
71
Posts
11
Kudos Received
15
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
892 | 07-19-2019 01:53 PM | |
12395 | 07-19-2019 12:17 PM | |
12409 | 07-19-2019 11:23 AM | |
851 | 07-15-2019 09:40 AM | |
632 | 04-04-2019 03:03 PM |
07-19-2019
01:53 PM
For every choice of semantics for these properties, we found failure scenarios that would result in good values overwriting bad values, either in CM during bootstrap or grow, or in Director's deployment or cluster template during refresh. The blacklist was the least-bad solution. The blacklisting will not result in errors if you provide the properties--they will just be silently ignored when Director passes the configuration to CM. With the blacklists in place, CM becomes the source of truth for these properties, so you will still be able to set them in CM if the autoconfigured values are not acceptable. If for some reason you have problems bootstrapping without them, the bootstrap and update blacklists are separately configurable, so you might be able to provide them at bootstrap. I can't recall what the potential failure scenarios were there.
... View more
07-19-2019
12:54 PM
Can you please clarify whether you were already using those custom settings and they are not being applied correctly, or whether you started providing those custom settings as a solution for the problem you raised in this post? If it's the latter, I'm glad you found the custom settings. That was going to be my recommendation. What version of Director are you on? While implementing improvements to the deployment/cluster refresh process in 6.2, we ran into some bugs around those settings getting overwritten during refresh. There is a new blacklist in place to avoid overwriting those properties (so that CMs autoconfigured values don't get smashed), but that could prevent your custom values from taking effect. So if you run into any problems on 6.2 or later, please let us know.
... View more
07-19-2019
12:17 PM
1 Kudo
According to the comments on the Jira, the properties will not include hikari. That is consistent with the recommendation in the previously linked stackoverflow thread.
... View more
07-19-2019
11:23 AM
1 Kudo
@Da It turns out that the inability to configure these properties was a Director bug which has been fixed for our upcoming 6.3 release. Once you are on 6.3, you will be able to configure the setting (for the Hikari connection pool, as it turns out) using the spring.datasource.maximumPoolSize property. I am sorry that there is no workaround for earlier releases.
... View more
07-19-2019
11:02 AM
We're relying on springboot, and are not doing anything special. I notice that in the application.properties embedded in the jar, some of the documented tomcat properties, like testOnBorrow and validationQuery, are being set with a prefix of spring.datasource.tomcat. Based on that stackoverflow answer, I wouldn't expect that to be required for maxActive, but give it a try and let me know if it works.
... View more
07-19-2019
09:32 AM
I can't confirm that your problem is caused by the connection pool size, and have never adjusted the connection pool settings manually, but Director uses spring-boot so you should be able to set the appropriate properties in application.properties as described in this stackoverflow question: https://stackoverflow.com/questions/25573034/spring-boot-how-do-i-set-jdbc-pool-properties-like-maximum-number-of-connection The property prefix should be spring.datasource, and the supported properties should be these: http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#Common_Attributes For the pool size specifically, you should be able to set it like this: spring.datasource.maxActive=200
... View more
07-18-2019
09:15 AM
It would be helpful for us to see the logs. There should be additional info logging for the individual connection attempts ("Attempting connection to <endpoint>").
... View more
07-15-2019
09:40 AM
It looks like we failed to update the documentation when we added support for this in 6.2.0. Good catch! The cluster refresher should detect changes to the instance type made in the Azure portal and update the cluster template with the new information. If this results in an instance group being inconsistent (with some instances on the old instance type and some on the new), grow operations will be blocked for the cluster. If the instance group is made consistent (by either changing all the instances to the new instance type, or by shrinking away instances using the old instance type), grow will be enabled, and a subsequent grow operation will use the new instance type.
... View more
04-04-2019
03:03 PM
Support for CentOS 7.6 was added in Cloudera Altus Director 6.2, which released today. Altus Director does not validate the CentOS version, but CentOS 7.6 has not been tested with older Altus Director versions.
... View more
07-26-2018
01:49 PM
Your prior information was clear. The mismatch was in my terminology. The term "deployment" is used in the APIs, while the UI says "Cloudera Manager". These are synonymous. Sorry for the confusion! Please see if you can terminate one of the failed Cloudera Managers, rather than one of the corresponding clusters. If that fails, I'm sorry to say that there is currently no safe way to force deletion of the deployment or cluster from Director's database. If this is purely a test setup, you could delete any successful deployments, stop the Director service, then start from scratch (on H2, by deleting the director database file from disk, or on MySQL, by creating a new empty database for Director with the appropriate permissions). When you start Director again, it will rebuild the database contents. If this is not feasible, you can just ignore the failed deployments and clusters.
... View more
07-26-2018
12:11 PM
There is a technical distinction in Cloudera Director between a deployment and a cluster. Each Cloudera Director instance can manage multiple environments (each corresponding to an account on a cloud provider), each of which can have multiple deployments (each corresponding to a single Cloudera Manager instance), each of which can have multiple clusters. I am specifically asking if you have tried terminating the deployment, rather than the cluster. In your last answer you still mentioned terminating the cluster.
... View more
07-26-2018
11:47 AM
Are there currently orphaned resources (such as instances) in Azure? If so, can you try deleting them manually and then trying to terminate the deployment via the Cloudera Director UI?
... View more
07-26-2018
11:44 AM
You mention that the deployment failed, and that you are trying to terminate the cluster. Have you tried terminating the deployment directly in the Cloudera Director UI?
... View more
07-25-2018
01:00 PM
The Cloudera Manager password can be updated in Cloudera Director in a variety of ways, which are not currently well-documented in the Cloudera Director User Guide. If you are using the Cloudera Director UI, you can use the "Update Cloudera Manager Credentials" option in the dropdown on the deployment page. A deployment update (PUT) through the API (using the API console or Java/Python SDKs) can also do it, since username and password are among the legal template fields to change.
... View more
07-24-2018
08:48 AM
You can safely ignore the "missing endpoint address string" warning. Do you have a stack trace or additional error message about the failure to reach Cloudera Manager? Cloudera Director must have SSH access to the CM instance and each of the cluster instances. You can achieve this by running Cloudera Director in the same subnet as the instances, by configuring some sort of VPN, or by configuring the instances to have public IP addresses.
... View more
06-21-2018
01:15 PM
Glad you figured this out. Thanks for posting your solution for other forum users.
... View more
06-20-2018
12:47 PM
I'm glad you figured this out. There are some other places you can look for information about provider-specific configuration properties as well. There are reference configuration files here: https://github.com/cloudera/director-scripts/tree/master/configs From the help menu in the Director UI, you can access the API console, and the provider-metadata endpoints there will show you all the valid provider-specific configuration properties. If you're comfortable reading code, the plugins that enable Director to work with multiple cloud providers are open source. If you search those repositories for ConfigurationPropertyToken, you will see the classes that define the metadata for the provider-specific configuration properties. The AWS repository, for example, is here: https://github.com/cloudera/director-aws-plugin/tree/v2
... View more
04-09-2018
09:03 AM
Glad your problem is solved. Thanks for letting the community know the solution. Hopefully if someone else ends up in your situation they will find your answer helpful.
... View more
11-13-2017
04:13 PM
Director's deployment and cluster refreshers should detect this sort of thing and update the deployment and cluster representations (respectively) in the database with the updated version information. Can you please check the server application.log to see if either of them encountered problems?
... View more
11-13-2017
04:10 PM
Yes, this is most likely a consequence of the other issue. We should try to address that one first.
... View more
11-03-2017
06:30 AM
The root cause should be a little earlier in the log. The log snippet you provided starts when Director is trying to clean up resources in response to the error.
... View more
10-04-2017
02:52 PM
1 Kudo
Yes. Are you attempting to specify a custom Name tag in your instance template? Cloudera Director by default uses that tag to attach the name that it generates for the instance. If it is not critical for you to provide a value for the Name tag, you should remove it from your configuration. If it is critical, it is possible to configure Director to use custom tag names. See the "Configuring Cloudera Director to Use Custom Tag Names on AWS" section of the Cloudera Director User Guide. There was a bug around this functionality in Director 2.5.0, but it should work correctly in Director 2.5.1 and later.
... View more
10-04-2017
01:13 PM
1 Kudo
I'm sorry you're having trouble getting your cluster running. You are trying to create the cluster through the Cloudera Director UI, right? If so (or if you're using the bootstrap-remote command from the command line), you should look in Cloudera Director server's application.log for an earlier exception giving the specific failure during instance allocation.
... View more
10-03-2017
09:52 AM
It turns out that the agent installation log is not currently part of the diagnostic data that Director tries to download from CM. If you do not get a useful error from the Director server log, you should SSH into the CM instance and search the /tmp directory for scm_prepare_node.log. There may be more than one of these because of the retries. Those log files may contain additional information about the reason for failure.
... View more
10-03-2017
09:15 AM
Try /var/log/cloudera-director-server/application.log. If it's not there use "sudo find . -name application.log" from some suitably high-level directory.
... View more
10-03-2017
09:08 AM
Have you already checked the Director server log for more details about the failure? In addition to looking for ERROR log messages and stack traces, you can look to see if Director was able to download diagnostic data from Cloudera Manager with additional information about the reason for agent installation failure.
... View more
08-23-2017
02:11 PM
This looks like a duplicate of your other post. Can you please confirm that this issue is resolved?
... View more
08-23-2017
02:10 PM
I'm glad your problem is resolved. Thank you for posting the details so other forum members can benefit from your experience.
... View more
08-23-2017
02:07 PM
Sorry for the delayed reply. Unfortunately, there is not enough info in that log snippet to determine why the agent installation failed. Do you know whether the agent installation failed on the CM instance or one of the cluster instances? If the former, look earlier in the Director log to see if there are any additional details about the specific failure. If the latter, also look in the CM log.
... View more