Support Questions
Find answers, ask questions, and share your expertise

How to deploy Spark2.0 on CDH5.9

Hi All,

 

I've successfully upgraded a development cluster to CDH5.9 with Parcels 5.9.0-1.cdh5.9.0.p0.23 activated. Now, I wanted to deploy Spark2.0 running side by side with 1.6 Spark facing issue. Here are steps.

 

1. Download the Spark2 csd from below.

http://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html

2. Move it to default /opt/cloudera/csd and set permissions accordingly. 

3. Restart both CM server and management services.

4. Spark2 shows up when try adding services but failed to start. When try deploying Client config and CM complaining additional Parcels need to be activated. 

 

When I look at below link latest parcels and only available for 5.7 for latest.

 

http://archive.cloudera.com/spark2/parcels/latest/

 

What am I missing here? where can I find additional parcels for Spark2.0 to download?

 

Your help or instruction is really appreciated.

 

Thanks,

Silaphet

 

2 REPLIES 2

Re: How to deploy Spark2.0 on CDH5.9

Master Collaborator

The parcel is actually fine for any version 5.7 or later.

The process worked for me on CDH 5.9.

 

What error about what parcel do you see?
You don't mention adding the parcel repo, and that's a step in the instructions. Did you do that? 

Re: How to deploy Spark2.0 on CDH5.9

Hi,

 

my wget command accidently downloaded old manifest.json and that was the reason CM was not able to detect it. Once I got correct manifest.json and everything was working as expected.

 

This has been fixed and thanks for your response.

 

Thanks,

Silaphet