Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
New Contributor

Microsoft HDInsights provides the four different options under Cluster Type and they are:

Hadoop, Hbase, Storm and Spark (Preview);

Some of HDP components are not standard in the HDInsight distribution of choice.

In this particular case, the customer was interested in Spark / Spark Notebooks Components and especially Zeppelin.

When choosing the Spark Option, only these services will be provided under Ambari:

HDFS, MR2, YARN, Tez, Hive, Pig, Sqoop, OOzie, Zookeeper, Ambari Metrics, Jupyter, Livy (Livy is the remote job submission server for Spark), and Spark;

Zeppelin is missing from the HDI Spark distro at the time of writing this article;

So in order to install Zeppelin When choosing 'Spark' as the HDI ‘Cluster Type' , and under

'New HDInsights Cluster' >> 'Optional Configuration’ >> 'Scripts Action' put the following URI as shown below:

https://hdiconfigactions.blob.core.windows.net/linuxincubatorzeppelinv01/install-zeppelin-spark151-v...

1592-zeppelin-actionscripts.png

Further services can be added for the Spark Cluster Type through Ambari (Add Service):

Accumulo, Atlas, Mahout, Ranger, Ranger KMS and Slider.

(notice that Hbase is not a default install when choosing the Spark HDI install option, and has to be installed seperatly or through chosing the Hadoop cluster Type instead of Spark)

Here are other 'Script Actions' URLs for installing Spark, R, Solr, and Giraph while installing HDInsight:

Install Spark >> https://hdiconfigactions.blob.core.windows.net/sparkconfigactionv03/spark-installer-v03.ps1.

Install R >> https://hdiconfigactions.blob.core.windows.net/rconfigactionv02/r-installer-v02.ps1.

Install Solr>> https://hdiconfigactions.blob.core.windows.net/solrconfigactionv01/solr-installer-v01.ps1.

Install Giraph >> https://hdiconfigactions.blob.core.windows.net/giraphconfigactionv01/giraph-installer-v01.ps1.

1,765 Views
Comments

@Alexander Do we have similar script to install Hbase on Spark HD Insights cluster ?

@Alexander is there a full list of these hdi scripts available? If not how did you discover the ones above?

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 01:27 PM
Updated by:
 
Contributors
Top Kudoed Authors