question Re: spark 2.2 parcel availability in CDH in Archives of Support Questions (Read Only)

spark 2.2 parcel availability in CDH

sudhirpatil — Fri, 16 Sep 2022 11:56:38 GMT

Hi,

Spark 2.2 is now realeased and is production ready. any estimate on when will the spark 2.2 will be available as CDH parcel? Any estimated/exact dates of parcel release is highly appriciated.

Thanks

Re: spark 2.2 parcel availability in CDH

mbigelow — Tue, 18 Jul 2017 22:34:52 GMT

http://community.cloudera.com/t5/Community-News-Release/ANNOUNCE-Cloudera-Distribution-of-Apache-Spark-2-2-Release-1/m-p/57600

Re: spark 2.2 parcel availability in CDH

MilesYao — Fri, 21 Jul 2017 20:29:21 GMT

That's good news. But I think the requester would like to know when Cloudera plans to integrate Spark 2 into CDH, not as a separate install (like what Hortonworks does).

Thanks,

Miles

Re: spark 2.2 parcel availability in CDH

srowen — Fri, 21 Jul 2017 20:33:03 GMT

It's as integrated and supported as anything else.

If you mean replacing Spark 1, no that's not possible as it would break any apps using Spark 1.

If you mean you just don't want a separate add-on, I get it, but, it's just a delivery mechanism.

C6 would not have both.

Re: spark 2.2 parcel availability in CDH

mbigelow — Fri, 21 Jul 2017 21:17:10 GMT

@MilesYao That may be. On that topic, I don't think anytime soon as Cloudera does not support many features that it does for Spark 1.6. I suspect that sometime post CDH 6 we will see Spark 2.x supplant Spark 1.x as the only version of Spark in CDH.

Ah, I checked out HDP and see what you are getting at. It is really trivial on the difference. Cloudera asks you to put a file on the CM host and configure a separate parcel while HDP includes both Spark1 and Spark2 packages in the same repo.