Community Articles
Find and share helpful community-sourced technical articles
Explorer

Microsoft HDInsights provides the four different options under Cluster Type and they are:

Hadoop, Hbase, Storm and Spark (Preview);

Some of HDP components are not standard in the HDInsight distribution of choice.

In this particular case, the customer was interested in Spark / Spark Notebooks Components and especially Zeppelin.

When choosing the Spark Option, only these services will be provided under Ambari:

HDFS, MR2, YARN, Tez, Hive, Pig, Sqoop, OOzie, Zookeeper, Ambari Metrics, Jupyter, Livy (Livy is the remote job submission server for Spark), and Spark;

Zeppelin is missing from the HDI Spark distro at the time of writing this article;

So in order to install Zeppelin When choosing 'Spark' as the HDI ‘Cluster Type' , and under

'New HDInsights Cluster' >> 'Optional Configuration’ >> 'Scripts Action' put the following URI as shown below:

https://hdiconfigactions.blob.core.windows.net/linuxincubatorzeppelinv01/install-zeppelin-spark151-v...

1592-zeppelin-actionscripts.png

Further services can be added for the Spark Cluster Type through Ambari (Add Service):

Accumulo, Atlas, Mahout, Ranger, Ranger KMS and Slider.

(notice that Hbase is not a default install when choosing the Spark HDI install option, and has to be installed seperatly or through chosing the Hadoop cluster Type instead of Spark)

Here are other 'Script Actions' URLs for installing Spark, R, Solr, and Giraph while installing HDInsight:

Install Spark >> https://hdiconfigactions.blob.core.windows.net/sparkconfigactionv03/spark-installer-v03.ps1.

Install R >> https://hdiconfigactions.blob.core.windows.net/rconfigactionv02/r-installer-v02.ps1.

Install Solr>> https://hdiconfigactions.blob.core.windows.net/solrconfigactionv01/solr-installer-v01.ps1.

Install Giraph >> https://hdiconfigactions.blob.core.windows.net/giraphconfigactionv01/giraph-installer-v01.ps1.

2,305 Views
Comments

@Alexander Do we have similar script to install Hbase on Spark HD Insights cluster ?

@Alexander is there a full list of these hdi scripts available? If not how did you discover the ones above?

Don't have an account?
Version history
Last update:
‎08-17-2019 01:27 PM
Updated by:
Contributors
Top Kudoed Authors