Reply
Expert Contributor
Posts: 61
Registered: ‎02-03-2016

CDH and Spark 2

I am curious about when CDH will officially move to Spark 2.x and no longer default to 1.6. Anyone have any information about this?

 

Thanks,

Ben

Cloudera Employee
Posts: 466
Registered: ‎08-11-2014

Re: CDH and Spark 2

CDH has been on Spark 2 for almost a year.

 

CDH5 can't remove Spark 1.x support because that would of course break any apps using Spark 1.

Spark 2 is provided in parallel.

 

Breaking changes like that would have to happen in major releases of the distro.

Expert Contributor
Posts: 61
Registered: ‎02-03-2016

Re: CDH and Spark 2

So, are you saying that CDH 6 would be the most likely version to move to Spark 2 entirely?

 

Cheers,

Ben

Highlighted
Cloudera Employee
Posts: 466
Registered: ‎08-11-2014

Re: CDH and Spark 2

That's a pretty good bet, though, I'm not a person that decides or knows what will be in C6.

Announcements