Reply
Explorer
Posts: 19
Registered: ‎12-02-2014

How to deploy Spark2.0 on CDH5.9

Hi All,

 

I've successfully upgraded a development cluster to CDH5.9 with Parcels 5.9.0-1.cdh5.9.0.p0.23 activated. Now, I wanted to deploy Spark2.0 running side by side with 1.6 Spark facing issue. Here are steps.

 

1. Download the Spark2 csd from below.

http://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html

2. Move it to default /opt/cloudera/csd and set permissions accordingly. 

3. Restart both CM server and management services.

4. Spark2 shows up when try adding services but failed to start. When try deploying Client config and CM complaining additional Parcels need to be activated. 

 

When I look at below link latest parcels and only available for 5.7 for latest.

 

http://archive.cloudera.com/spark2/parcels/latest/

 

What am I missing here? where can I find additional parcels for Spark2.0 to download?

 

Your help or instruction is really appreciated.

 

Thanks,

Silaphet

 

Cloudera Employee
Posts: 481
Registered: ‎08-11-2014

Re: How to deploy Spark2.0 on CDH5.9

The parcel is actually fine for any version 5.7 or later.

The process worked for me on CDH 5.9.

 

What error about what parcel do you see?
You don't mention adding the parcel repo, and that's a step in the instructions. Did you do that? 

Highlighted
Explorer
Posts: 19
Registered: ‎12-02-2014

Re: How to deploy Spark2.0 on CDH5.9

Hi,

 

my wget command accidently downloaded old manifest.json and that was the reason CM was not able to detect it. Once I got correct manifest.json and everything was working as expected.

 

This has been fixed and thanks for your response.

 

Thanks,

Silaphet