Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CDH and Spark 2

Highlighted

CDH and Spark 2

Rising Star

I am curious about when CDH will officially move to Spark 2.x and no longer default to 1.6. Anyone have any information about this?

 

Thanks,

Ben

3 REPLIES 3

Re: CDH and Spark 2

Master Collaborator

CDH has been on Spark 2 for almost a year.

 

CDH5 can't remove Spark 1.x support because that would of course break any apps using Spark 1.

Spark 2 is provided in parallel.

 

Breaking changes like that would have to happen in major releases of the distro.

Re: CDH and Spark 2

Rising Star

So, are you saying that CDH 6 would be the most likely version to move to Spark 2 entirely?

 

Cheers,

Ben

Re: CDH and Spark 2

Master Collaborator

That's a pretty good bet, though, I'm not a person that decides or knows what will be in C6.