Give Azure Data Factory (ADF) a try. ... View more

Use an HDInsight on Linux cluster to have control over VMs through Ambari. Take a look at the public preview of Azure Data Lake Store which gets you past the storage account throttling and total size limits. Because you have separation of storage and compute you can move and load data with tools outside the cluster (SSIS, AzCopy, ADF, etc.), even when the cluster doesn't currently exist. Multiple clusters of HDInsight plus Azure Data Lake Analytics can all access the same data at the same time. ... View more

HDP on Azure VMs (IaaS) is available from the Azure Marketplace. ... View more

Blob storage & (public preview) Azure Data Lake store are the options for HDFS in HDInsight. They allow the separation of storage & compute that enables all the scale, flexibility, and cost savings available in the cloud. If you have a specific use case where attaching an existing VHD or SSD would be useful please reach out: cgross@microsoft.com. We have multiple versions of HDInsight available, each has different versions of HDP components. As a general rule you will see three versions of HDInsight at any given time to enable both backwards compatibility and the newest releases ... View more

Generally choose HDInsight on Linux. This enables Ambari actions & lots of third party code. To see all VM type choices for HDInsight click the "show all" button. Subscriptions have a quota/billing limit, you can request to increase your quota of HDInsight cores with a "billing support" ticket. You will enter how many HDInsight Linux cores you want per Azure region (i.e. US West or China). ... View more

Once you're on Azure you have access to Azure Data Factory (ADF) and Azure Data Lake Store (ADLS) as well. ADF is a workflow orchestration tool that works with on-premises and cloud data pipelines. ADLS is an HDFS in the cloud storage solution that enables data sharing via Azure Active Directory authentication across multiple Hadoop clusters and other HDFS-compliant computes. ... View more