Reply
Explorer
Posts: 10
Registered: ‎04-26-2016

Can I install Spark 2 on the QuickStart VM?

I'm interested in learning Spark, but I'm of the impression that it would be better to focus on learning based upon Spark 2, instead of Spark 1.x.  

 

Spark 1.6 is part of the current Quickstart VM (v5.12), but I was wondering if it was possible to install Spark 2 instead?  I migrated to Parcels and enabled Cloudera Manager on the Quickstart VM, but I don't see Spark 2. 

 

Is there a way of installing Spark 2 on the Quickstart VM?

Master
Posts: 305
Registered: ‎07-01-2015

Re: Can I install Spark 2 on the QuickStart VM?

Yes, just need to download a parcel for Spark2, activate it, and then go to Services -> Add new service.
Champion
Posts: 744
Registered: ‎05-16-2016

Re: Can I install Spark 2 on the QuickStart VM?

this is the parcel location to download 

http://archive.cloudera.com/beta/spark2/parcels/2.0/

 

please refer the below url for the walkthrough 

https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html

 

refer the screenshots. 

please let me know if you need more information. 

Champion
Posts: 744
Registered: ‎05-16-2016

Re: Can I install Spark 2 on the QuickStart VM?

screen1.png

 

 

 

screen2.png

New Contributor
Posts: 2
Registered: ‎06-29-2016

Re: Can I install Spark 2 on the QuickStart VM?

for Cloudera manager, it needs minimum 8GB RAM. Is there any way I can replace current spark 1.6 version to 2 without using parcel so that I can run the VM with less than 8 GB?

Explorer
Posts: 10
Registered: ‎04-26-2016

Re: Can I install Spark 2 on the QuickStart VM?

Unfortunately I was never able to get this working.  The cluster would crash, things wouldn't work, etc... I'm going to sit this one out for now and just wait until an updated QuickStart VM comes up with Spark 2.  I'll just use a different distro. for now for Spark 2.

Explorer
Posts: 8
Registered: ‎05-19-2017

Re: Can I install Spark 2 on the QuickStart VM?

ERROR:

CDH 5.12.0-1.cdh5.12.0.p0.29 conflicts with SPARK2 2.0.0.cloudera.beta2-1.cdh5.7.0.p0.110234.

 

Screenshot - 6_17_2018 , 3_01_13 PM.jpg

Announcements