question Recommendations for max number of deployments / clusters director should manage in Support Questions

Recommendations for max number of deployments / clusters director should manage

dturner — Fri, 16 Sep 2022 15:34:02 GMT

Assuming the latest version of Altus Director... are there any documentated recommendations for the number of deployments/clusters that Director can reasonbly be expected to manage?

Our clusters are less than 21 nodes each and our Director server is running on an EC2 instance with 8 cores and 60GB of RAM and I have a dedicated RDS MariaDB DB sized at 2 cores and 16GB of RAM with 200GB of storage. Thus far I'm managing around 14 clusters from ranging in size from 9 to 20 nodes. No issues thus far. I'm just curious if there is general guidance around this subject.

Re: Recommendations for max number of deployments / clusters director should manage

dhan — Thu, 11 Jul 2019 15:49:32 GMT

dturner,

We don't have any documented recommendations for the max deployments or clusters. Anecdotally, I've heard of 100+ clusters. Director does the most work when bootstrapping/updating/terminating clusters and less when just monitoring clusters.

You are also running on a way larger instance than we usually use so I bet there's room for some tuning if you do run into issues (e.g., increasing memory, threadpools, etc).

Re: Recommendations for max number of deployments / clusters director should manage

dturner — Thu, 11 Jul 2019 15:55:36 GMT

Thank you for the response.

To your point about bootstrapping/terminating - I have noticed during testing when I'm cycling through many bootstrapping / terminations that Director can get into a state where it seems to no longer respond to bootstrapping requests (from command line). I'll typically let Director "cool down" for a few minutes and retry.

Re: Recommendations for max number of deployments / clusters director should manage

Da — Thu, 18 Jul 2019 16:03:42 GMT

From experience Director will really lag as you scale higher up. We are at 5 environments with about 40 deployments spanning thousand of nodes.

Make sure you go into application.properties and increase the amount of threads for tasks. That really makes a difference from what I've been seeing.