Reply
New Contributor
Posts: 3
Registered: ‎02-24-2016
Accepted Solution

Spark 2

CDH (lower than 5.12) parcel required for SPARK2 (2.0.0.cloudera1-1.cdh5.7.0.p0.113931) is not available. How to solve this problem? I already copy SPARK2_ON_YARN-2.0.0.cloudera1.jar to the system and follow all steps from manuals.
Expert Contributor
Posts: 82
Registered: ‎02-24-2016

Re: Spark 2

Did you find any solution
Cloudera Employee
Posts: 481
Registered: ‎08-11-2014

Re: Spark 2

There is no CDH 5.12. Spark 2 is availabled as a CSD. Please follow the documented steps for installing it, which never includes manually copying JARs. http://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html

Expert Contributor
Posts: 82
Registered: ‎02-24-2016

Re: Spark 2

[ Edited ]

Followed same institutions. Downloaded CSD jar file and moved to /opt/cloudera/csd and restarted the server and management services. But no luck

Cloudera Employee
Posts: 481
Registered: ‎08-11-2014

Re: Spark 2

It sounds like you only finished step 2. You need to finish all of them.

New Contributor
Posts: 3
Registered: ‎02-24-2016

Re: Spark 2

"4.Download the Spark2 parcel, distribute the parcel to the hosts in your cluster, and activate the parcel. See Managing Parcels." I have the trouble with activation.
Expert Contributor
Posts: 82
Registered: ‎02-24-2016

Re: Spark 2

[ Edited ]

@sroweni did all, downloaded parcel and distributed to all nodes, while activating the parcel i'm getting CDH (lower than 5.12) parcel required for SPARK2 (2.0.0.cloudera1-1.cdh5.7.0.p0.113931) is not available

New Contributor
Posts: 3
Registered: ‎02-24-2016

Re: Spark 2

by Victor 

figured it out, under parcel configurations, if I uncheck
Validate Parcel Relations

Expert Contributor
Posts: 82
Registered: ‎02-24-2016

Re: Spark 2

[ Edited ]

I hope it's not a good idea, as it says "Enforce that parcel dependencies are satisfied and conflicts are prevented when activating parcels. Parcel relations (Depends, Conflicts, and Replaces) can be defined the manifests of parcel repositories. Cloudera Manager can also enforce some default relations if none are defined in the manifest"

 

Anyway thanks for the tip :-)

Cloudera Employee
Posts: 97
Registered: ‎05-10-2016

Re: Spark 2

Yes, this would typically not be recommened, but is a work around for a bug.  This is fixed in CM 5.9, so when using 5.9 and newer, you should not need to disable parcel relation validation.