question Ease of scalability with HDInsight in Support Questions

Ease of scalability with HDInsight

vivek_sharma — Wed, 28 Dec 2016 17:03:59 GMT

Is auto scaling available with HDInsight on Azure?

As per HDCloud documentation, HDloud has two auto scaling groups (one for the master node and another one for the worker nodes) and thus is auto scalable. Is some thing similar available with HDInsight or we have to manually scale the clusters.

As per the HDinsight documentation all HDInsight clusters allow to change the number of nodes in the cluster. I could not explicitly find if it is auto scalable.

Re: Ease of scalability with HDInsight

mpandit — Thu, 29 Dec 2016 05:52:09 GMT

Have you checked this one https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-administer-use-management-portal

Re: Ease of scalability with HDInsight

tmccuch — Tue, 03 Jan 2017 05:46:51 GMT

@Vivek Sharma

With Cloudbreak Periscope, you can define a scaling policy and apply it to any Alert on any Ambari Metric. Scaling granularity is at the Ambari host group level. This feature, which we refer to as auto-scaling, is only a capability of Cloudbreak at this point in time. Per your line of questioning above, if you use Cloudbreak to provision HDP on either Azure IaaS or AWS IaaS, you can use the auto-scaling capabilities it provides.

Both Azure HDInsight (HDI) and Hortonworks Data Cloud for AWS (HDC) make it very easy to manually re-size your cluster through their respective consoles. However, the auto-scaling feature described above is not available with either HDI or HDC at this point in time.

Re: Ease of scalability with HDInsight

tmccuch — Wed, 04 Jan 2017 01:46:37 GMT

@Vivek Sharma If this answer helps, please accept it. Otherwise, I'd be happy to answer any remaining questions you have. Thanks! _Tom