Support Questions
Find answers, ask questions, and share your expertise

Installing Spark2

Explorer

http://archive.cloudera.com/spark2/parcels/2.0.0.cloudera2/

 

How do I add this to my cluster conf files to install Spark2?

1 ACCEPTED SOLUTION

Accepted Solutions

Explorer

 

 

I was able to add it as a service after activating the parcel AND downloading the csd jar. I didn't realize that I needed both of these. I thought it was either or. 

View solution in original post

9 REPLIES 9

Champion

@ujj

 

CM -> Administrator -> Settings -> Custom Service descriptor (CSD) Path
CM -> Hosts -> Parcel

 

Navigate to Custom Service Descriptor (CSD) & Parcel to understand the available options and refer the below link, it will help you to add the service

 

https://www.cloudera.com/documentation/enterprise/5-5-x/topics/cm_mc_addon_services.html

Explorer
I can't automate it at cluster bootstrap time?

Explorer

 

Went there, activated it, but I still don't see it as a choice when I Add Service. 

Expert Contributor

Hi wkupersa,

 

Can you fully describe your scenario? Are you attempting to use Cloudera Director to create a new cluster including Spark 2, or do you have an existing cluster you'd like to add Spark 2 to? Or is it some other scenario? The additional information will help narrow down the possible issues.

 

Thanks,

Bill

Explorer

Thanks Bill,

 

I just built this cluster using CDH 5.11.0.  I installed Spark1.6.0 thought the wizard along with Yarn, ZooKeeper, and HDFS. I verified that Spark 1.6.0 worked and later added Hive as well.  I added the parcel configuration for Spark 2. I downloaded it, distributed it, and activated it. It appears as distributed and anctivated under parcels.  I then expected to see it in the list of services that I could add under the cluster's  "Add Service" option, but I don't.  I turned off "Validate Parcel Relations" to see if that would cause it to appear, but it didn't. 

 

 

Explorer

 

 

I was able to add it as a service after activating the parcel AND downloading the csd jar. I didn't realize that I needed both of these. I thought it was either or. 

View solution in original post

Cloudera Employee

That's correct, cool that you solved it 🙂 A simple way to think about it is that a parcel contains the binary contents, and a CSD contains the metadata for a service.

New Contributor
All operations are usefully.But do you restart the CM & Cloudera Manager Agent? And You will deploy the Client Configuration after above tasks. Even if you will have to upgrade the jre version.

Explorer

Hi,

 

I have follwed all the steps to install Spark2 in cloudera vm but getting following error when I am trying to Host spark2 parcel.

 

CDH (5.8 and higher) parcel required for SPARK2 (2.2.0.cloudera2-1.cdh5.12.0.p0.232957) is not available.

 

I am using CDH version : 5.12 and using http://archive.cloudera.com/spark2/parcels/2.2.0.cloudera2/  link for parcel.

 

Please help me with the configurtion.