Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Repos and Manual Install Instructions for Spark 2.0 on top of CDH 5.11

Repos and Manual Install Instructions for Spark 2.0 on top of CDH 5.11

Explorer

Hi,

I build a pseudo-cluster CDH install in a VM for my students each Fall.  Due to the limitations of some of their laptops, I try to keep the footprint as small as possible.
For that reason, I manually install CDH rather than use Cloudera Manager. The cloudera documentation usually does a nice job of laying out the steps and this approach has been very successful for me.

I am upgrading the VM this summer to be based on Centos 7, CDH 5.11 and also Spark 2.

I have had the hardest time trying to track down the manual instructions using the Cloudera repository to install Spark 2.

The cloudera documentation describes how to accomplish the Spark 2.0 install using parcels and Cloudera Manager, but nothing on the correct cloudera repository and a manual install using yum.

Any advice would be greatly welcome.

rd

2 REPLIES 2

Re: Repos and Manual Install Instructions for Spark 2.0 on top of CDH 5.11

Master Guru
Unfortunately, currently Spark 2 is only available in parcel form:

"""
Cloudera Distribution of Apache Spark 2 is distributed as two files: a CSD file and a parcel, both of which need to be installed on the cluster.
""" - https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html

Re: Repos and Manual Install Instructions for Spark 2.0 on top of CDH 5.11

New Contributor
Are Spark 2 as RPM packages on the roadmap? Any timeframe you could share?