About Da

Da · ‎09-26-2019

After upgrading to 6.3 and setting the following in the application.properties: spring.datasource.minimumIdle=100 spring.datasource.maximumPoolSize=200 I can see in the DEBUG logs of HikariConfig that those settings are being respected and the cloudwatch charts show the same. Thanks again!

Da · ‎09-26-2019

Thanks for this info (and link) and duly noted on troubleshooting in the future!

Da · ‎07-29-2019

Hey Ben, Thanks for getting back to me and sorry about the delay in responding. I ended up testing this Friday and everything went smoothly with the CSD. I did run into some issues with a custom parcel and CM making the wrong *.sha file upon downloading but putting the parcel in /opt/cloudera/parcel-repo helped clear that up. I may end up making that a different forum post as it seems a bit buggy. Regards, Dan

Da · ‎07-24-2019

@Mike Wilson On that note I managed to find another issue with the validator. If you create a VirtualInstanceGroup similar to the python example here: https://github.com/cloudera/director-sdk/blob/master/python-client-samples/cluster.py#L159 And then change the name in the VirtualInstanceGroup to masters.1 (add an invalid character to it like a .) it will create the cluster successfully but then the user in the UI will not be able to repair nodes / grow / shrink nor clone the cluster. It will also just gray out the continue button and provide no feedback to the user in the logs nor the UI. Cheers!

Da · ‎07-23-2019

This is because the limit file (/proc/<director_pid>/limit) of the process has a "Max open files" of 1024 which is to low for most operations. A solution for this since it uses systemd on RHEL/CentOS 7 is to do the following: # make a folder for custom systemd changes for this service mkdir -p /etc/systemd/system/cloudera-director-server.service.d/ # make an override conf file so that a Director upgrade will not break the changes vim /etc/systemd/system/cloudera-director-server.service.d/override.conf # then add the following in that file and save/quit it [Service] LimitNOFILE=65536 # next reload the daemon systemctl daemon-reload # finally restart Director systemctl restart cloudera-director-server Then if you check the limit file in the new process you will see it show 65536 as the "Max open files". Hopefully this can help someone in the future. Cheers!

Da · ‎07-22-2019

As it turns out this was my screwup -- I tried to get the template (to verify it's existence) with the InstanceTemplate object instead of the string name before creating it with my defined InstanceTemplate object. Apologies for any inconvenience!

Da · ‎07-22-2019

Alright awesome. Thanks for providing clarity regarding what the expected behavior will be!

Da · ‎07-18-2019

From experience Director will really lag as you scale higher up. We are at 5 environments with about 40 deployments spanning thousand of nodes. Make sure you go into application.properties and increase the amount of threads for tasks. That really makes a difference from what I've been seeing.

Online	Offline
Last Visited	‎03-06-2020 09:17 PM

Member Since	‎05-14-2019 02:24 PM
Last Visited	‎03-06-2020 09:17 PM
Posts	26

Cloudera Community

Re: Too many open files

Re: "A template must be specified"

Re: The request was rejected because the URL conta...

Re: How to increase database pool for Director?

Re: API call to Cloudera Manager failed for Cloude...

Re: Custom CSDs from CM 5.x to 6.x

Re: "A template must be specified"

Re: Too many open files

Re: The request was rejected because the URL conta...

Re: Director Not Increasing Heap of Host Monitor

Re: Recommendations for max number of deployments ...