Reply
Highlighted
ujj
Explorer
Posts: 7
Registered: ‎05-16-2016
Accepted Solution

Installing Spark2

http://archive.cloudera.com/spark2/parcels/2.0.0.cloudera2/

 

How do I add this to my cluster conf files to install Spark2?

Posts: 258
Topics: 11
Kudos: 40
Solutions: 25
Registered: ‎09-02-2016

Re: Installing Spark2

@ujj

 

CM -> Administrator -> Settings -> Custom Service descriptor (CSD) Path
CM -> Hosts -> Parcel

 

Navigate to Custom Service Descriptor (CSD) & Parcel to understand the available options and refer the below link, it will help you to add the service

 

https://www.cloudera.com/documentation/enterprise/5-5-x/topics/cm_mc_addon_services.html

ujj
Explorer
Posts: 7
Registered: ‎05-16-2016

Re: Installing Spark2

I can't automate it at cluster bootstrap time?
Explorer
Posts: 14
Registered: ‎01-14-2017

Re: Installing Spark2

 

Went there, activated it, but I still don't see it as a choice when I Add Service. 

Cloudera Employee
Posts: 35
Registered: ‎02-18-2014

Re: Installing Spark2

Hi wkupersa,

 

Can you fully describe your scenario? Are you attempting to use Cloudera Director to create a new cluster including Spark 2, or do you have an existing cluster you'd like to add Spark 2 to? Or is it some other scenario? The additional information will help narrow down the possible issues.

 

Thanks,

Bill

Explorer
Posts: 14
Registered: ‎01-14-2017

Re: Installing Spark2

Thanks Bill,

 

I just built this cluster using CDH 5.11.0.  I installed Spark1.6.0 thought the wizard along with Yarn, ZooKeeper, and HDFS. I verified that Spark 1.6.0 worked and later added Hive as well.  I added the parcel configuration for Spark 2. I downloaded it, distributed it, and activated it. It appears as distributed and anctivated under parcels.  I then expected to see it in the list of services that I could add under the cluster's  "Add Service" option, but I don't.  I turned off "Validate Parcel Relations" to see if that would cause it to appear, but it didn't. 

 

 

Explorer
Posts: 14
Registered: ‎01-14-2017

Re: Installing Spark2

 

 

I was able to add it as a service after activating the parcel AND downloading the csd jar. I didn't realize that I needed both of these. I thought it was either or. 

Cloudera Employee
Posts: 6
Registered: ‎01-30-2017

Re: Installing Spark2

That's correct, cool that you solved it :) A simple way to think about it is that a parcel contains the binary contents, and a CSD contains the metadata for a service.

Announcements