Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar
Rising Star

In terms of Azure HDInsight environment, here are few things to be aware of in terms of infrastructure:

  • You have option to install HDInsight on Windows or HDInight on Linux (only on ubuntu 12 LTS). Apache Ambari only comes only with Linux based install.
  • Type of machines used for Linux-based install were limited to D3, D4 & D12. Not sure if this is because of my Azure account limitations.
  • HDInsight version is 3.2.1 which comes with HDP 2.2 certain components.
  • Separate cluster required for Hadoop, Hbase , Storm. And Spark is available as Technical preview.
  • Uses Blob storage as default for HDFS. Not sure if there is a option to add VHD or SSD.
  • HDInsight 3.2 does not contain Falcon, Flume, Accumulo, Ambari Metrics, Atlas, Kafka, Knox, Ranger, Ranger KMS & Slider. Also it has a bit older version of hadoop components.

Attached is a file that has comparison of HDInsight 3.2.1 components to that of HDP 2.3.2.

hdinsight-and-hdp-component-comparison.zip

Update by @Ancil McBarnett

HDInsight Component Versioning: https://azure.microsoft.com/en-us/documentation/articles/hdinsight-component-versioning/

9,215 Views
Comments
avatar
Master Mentor

Thanks @Chakra

avatar

Good insight chakra

avatar
Explorer
  • Generally choose HDInsight on Linux. This enables Ambari actions & lots of third party code.
  • To see all VM type choices for HDInsight click the "show all" button. Subscriptions have a quota/billing limit, you can request to increase your quota of HDInsight cores with a "billing support" ticket. You will enter how many HDInsight Linux cores you want per Azure region (i.e. US West or China).
avatar
Explorer
  • Blob storage & (public preview) Azure Data Lake store are the options for HDFS in HDInsight. They allow the separation of storage & compute that enables all the scale, flexibility, and cost savings available in the cloud. If you have a specific use case where attaching an existing VHD or SSD would be useful please reach out: cgross@microsoft.com.
  • We have multiple versions of HDInsight available, each has different versions of HDP components. As a general rule you will see three versions of HDInsight at any given time to enable both backwards compatibility and the newest releases
avatar
Explorer

HDP on Azure VMs (IaaS) is available from the Azure Marketplace.

avatar
Expert Contributor

Thanks for the document and high level comparison. Do you have cost comparision also if we run HDI on monthly basis for 30 days against same of HDP on Azure with similar configuration?

avatar
Contributor

Hi All .

 

here is more Details about above :-

 

https://community.cloudera.com/t5/Support-Questions/HDInsight-Vs-HDP-Service-on-Azure-Vs-HDP-on-Azur...

 

 

 

Thanks

HadoopHelp

Version history
Last update:
‎10-27-2015 12:11 PM
Updated by:
Contributors