spark 2.2 parcel availability in CDH

[#1]

Hi,

Spark 2.2 is now realeased and is production ready. any estimate on when will the spark 2.2 will be available as CDH parcel? Any estimated/exact dates of parcel release is highly appriciated.

Thanks

[#2 · Accepted solution]

http://community.cloudera.com/t5/Community-News-Release/ANNOUNCE-Cloudera-Distribution-of-Apache-Spa...

[#3]

That's good news. But I think the requester would like to know when Cloudera plans to integrate Spark 2 into CDH, not as a separate install (like what Hortonworks does). 

Thanks,

Miles

[#4]

It's as integrated and supported as anything else.

If you mean replacing Spark 1, no that's not possible as it would break any apps using Spark 1.

If you mean you just don't want a separate add-on, I get it, but, it's just a delivery mechanism.

C6 would not have both.

[#5]

@MilesYao That may be. On that topic, I don't think anytime soon as Cloudera does not support many features that it does for Spark 1.6. I suspect that sometime post CDH 6 we will see Spark 2.x supplant Spark 1.x as the only version of Spark in CDH.

Ah, I checked out HDP and see what you are getting at. It is really trivial on the difference. Cloudera asks you to put a file on the CM host and configure a separate parcel while HDP includes both Spark1 and Spark2 packages in the same repo.