HDInsight Component Comparison to HDP

[#1]

In terms of Azure HDInsight environment, here are few things to be aware of in terms of infrastructure:

• You have option to install HDInsight on Windows or HDInight on Linux (only on ubuntu 12 LTS). Apache Ambari only comes only with Linux based install.
• Type of machines used for Linux-based install were limited to D3, D4 & D12. Not sure if this is because of my Azure account limitations.
• HDInsight version is 3.2.1 which comes with HDP 2.2 certain components.
• Separate cluster required for Hadoop, Hbase , Storm. And Spark is available as Technical preview.
• Uses Blob storage as default for HDFS. Not sure if there is a option to add VHD or SSD.
• HDInsight 3.2 does not contain Falcon, Flume, Accumulo, Ambari Metrics, Atlas, Kafka, Knox, Ranger, Ranger KMS & Slider. Also it has a bit older version of hadoop components.

Attached is a file that has comparison of HDInsight 3.2.1 components to that of HDP 2.3.2.

hdinsight-and-hdp-component-comparison.zip

Update by @Ancil McBarnett

HDInsight Component Versioning: https://azure.microsoft.com/en-us/documentation/articles/hdinsight-component-versioning/

[#2]

Thanks @Chakra

[#3]

Good insight chakra

[#4]

• Generally choose HDInsight on Linux. This enables Ambari actions & lots of third party code.
• To see all VM type choices for HDInsight click the "show all" button. Subscriptions have a quota/billing limit, you can request to increase your quota of HDInsight cores with a "billing support" ticket. You will enter how many HDInsight Linux cores you want per Azure region (i.e. US West or China).

[#5]

• Blob storage & (public preview) Azure Data Lake store are the options for HDFS in HDInsight. They allow the separation of storage & compute that enables all the scale, flexibility, and cost savings available in the cloud. If you have a specific use case where attaching an existing VHD or SSD would be useful please reach out: [email protected].
• We have multiple versions of HDInsight available, each has different versions of HDP components. As a general rule you will see three versions of HDInsight at any given time to enable both backwards compatibility and the newest releases

[#6]

HDP on Azure VMs (IaaS) is available from the Azure Marketplace.

[#7]

Updated HDInsight 3.3 component link is at

https://azure.microsoft.com/en-us/documentation/articles/hdinsight-component-versioning/

[#8]

Thanks for the document and high level comparison. Do you have cost comparision also if we run HDI on monthly basis for 30 days against same of HDP on Azure with similar configuration?

[#9]

Hi All .

here is more Details about above :-

https://community.cloudera.com/t5/Support-Questions/HDInsight-Vs-HDP-Service-on-Azure-Vs-HDP-on-Azur...

Thanks

HadoopHelp