Reply
Highlighted
Explorer
Posts: 16
Registered: ‎03-15-2015

Repos and Manual Install Instructions for Spark 2.0 on top of CDH 5.11

Hi,

I build a pseudo-cluster CDH install in a VM for my students each Fall.  Due to the limitations of some of their laptops, I try to keep the footprint as small as possible.
For that reason, I manually install CDH rather than use Cloudera Manager. The cloudera documentation usually does a nice job of laying out the steps and this approach has been very successful for me.

I am upgrading the VM this summer to be based on Centos 7, CDH 5.11 and also Spark 2.

I have had the hardest time trying to track down the manual instructions using the Cloudera repository to install Spark 2.

The cloudera documentation describes how to accomplish the Spark 2.0 install using parcels and Cloudera Manager, but nothing on the correct cloudera repository and a manual install using yum.

Any advice would be greatly welcome.

rd

Posts: 1,565
Kudos: 286
Solutions: 239
Registered: ‎07-31-2013

Re: Repos and Manual Install Instructions for Spark 2.0 on top of CDH 5.11

Unfortunately, currently Spark 2 is only available in parcel form:

"""
Cloudera Distribution of Apache Spark 2 is distributed as two files: a CSD file and a parcel, both of which need to be installed on the cluster.
""" - https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html
Backline Customer Operations Engineer
New Contributor
Posts: 1
Registered: ‎11-09-2017

Re: Repos and Manual Install Instructions for Spark 2.0 on top of CDH 5.11

Are Spark 2 as RPM packages on the roadmap? Any timeframe you could share?
Announcements