Ease of scalability with HDInsight

[#1]

Is auto scaling available with HDInsight on Azure?

As per HDCloud documentation, HDloud has two auto scaling groups (one for the master node and another one for the worker nodes) and thus is auto scalable. Is some thing similar available with HDInsight or we have to manually scale the clusters.

As per the HDinsight documentation all HDInsight clusters allow to change the number of nodes in the cluster. I could not explicitly find if it is auto scalable.

[#2]

Have you checked this one https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-administer-use-management-portal

[#3 · Accepted solution]

@Vivek Sharma

With Cloudbreak Periscope, you can define a scaling policy and apply it to any Alert on any Ambari Metric. Scaling granularity is at the Ambari host group level. This feature, which we refer to as auto-scaling, is only a capability of Cloudbreak at this point in time. Per your line of questioning above, if you use Cloudbreak to provision HDP on either Azure IaaS or AWS IaaS, you can use the auto-scaling capabilities it provides.

Both Azure HDInsight (HDI) and Hortonworks Data Cloud for AWS (HDC) make it very easy to manually re-size your cluster through their respective consoles. However, the auto-scaling feature described above is not available with either HDI or HDC at this point in time.

[#4]

@Vivek Sharma If this answer helps, please accept it. Otherwise, I'd be happy to answer any remaining questions you have. Thanks! _Tom