Cloudera Director does not currently support creating multi-AZ instances.
When this will be available ? is it possible to modify CF template to achieve this.
This feature is on our roadmap, but we don't have a planned release date for it yet.
In the meanwhile, I confirmed with one of our engineers - it is possible to create a cluster that will have instance in multiple AZs today by using multiple instance templates (with different subnets) but Director will not take any action to configure rack information.
Hope this helps,
Thanks Jayita for great help so far.
what is rack information that director won't be able to configure.
It is a Cloudera Manager configuration, that would need to be updated to handle multiple AZs.
Please note that this is not a supported or documented feature at this time. Once we release support for it, there will be additional details available.
You have already answered before we can bring up cluster by changing subnets but some issue with rack information with cloudera manager.
Can you please re-confirm issue related with rack/CM, if it is fixed or can we live with it by suppressing it, if no major impact.
If we setup multi-AZ based CDH by supplying two different subnets from different az each time.
Here's the Cloudera Manager documentation on racks.
The issue with running on multiple AZ is that increased network latency may affect data transfer or job performance. Rack awareness is used to mitigate this by running jobs closer to where the data lives.
You can assign racks to your hosts in CM after Director bootstraps or updates your cluster but there is no way to set racks through Director at this point.
If you have metrics for your desired performance level, then there's no replacement for measuring performance on an actual cluster as latency will be different depending on which AZ you choose.
Our major workload is mainly OLTP over Solr & Hbase. What would be your view on keeping Solr shards on two AZs or Hbase region server running over two AZs?