question Created local repository for CDH 6.2.0 but unable to get Spark-2 now in Support Questions

Created local repository for CDH 6.2.0 but unable to get Spark-2 now

Ansar_2601 — Fri, 16 Sep 2022 14:35:16 GMT

Hi Team,

I have installed CDH 6.2.0 on CM 6.2.0 with the help of local repository parcel file created by me following the Cloudera documents ( https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ig_create_local_parcel_repo.html ). I have successfully installed the CDH. however, CDH 6.2 comes with in-built parcels for Spark-2. which I am unable to find in the parcels.

Please assist me on how do I add Spark-2 as a service in my Cloudera 6.2.0 now.

Appreciate your assistance.

Re: Created local repository for CDH 6.2.0 but unable to get Spark-2 now

venkatsambath — Wed, 26 Feb 2020 12:12:52 GMT

Hi Ansar,

For CDH5.x version the spark service that is bundled with the CDH parcel was spark 1.6 version hence there was a need to add Spark2 as a separate parcel.

You need not add spark2 separately for CDH6.2. When you just add spark service in CDH6.2, it will install spark2.4.0 -- For your reference, This link contains the individual service packaged inside cdh6.2.x https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_62_packaging.html#cdh_621_packaging

If you would like to validate the version of spark installed alternatively you can run spark-shell --> This will show the spark version like this

Spark session available as 'spark'. Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 2.4.0-cdh6.3.3 /_/ Using Scala version 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_141) Type in expressions to have them evaluated.

Re: Created local repository for CDH 6.2.0 but unable to get Spark-2 now

Ansar_2601 — Wed, 26 Feb 2020 13:08:09 GMT

Thanks @venkatsambath

That helped.

Regards,
Ansar