Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Adding Spark 2.2

Solved Go to solution
Highlighted

Adding Spark 2.2

New Contributor

Hi,

We have an Ambari cluster with HDFS, YARN and Map reduce 2 all at version 2.7.3

There is a built in option of adding Spark 1.6.3 or 2.1.1 to the cluster, but we are looking to install 2.2

Have anyone tried Spark 2.2 here yet, or is there an upgrade option we are not aware of?

Thanks!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Adding Spark 2.2

Hi @Aviram Voda

Spark 2.2 is not yet supported with HDP 2.6.x that's why it's not available through Ambari (Install/upgrade). It has to go through our QE to be added in further release.

View solution in original post

5 REPLIES 5
Highlighted

Re: Adding Spark 2.2

Hi @Aviram Voda

Spark 2.2 is not yet supported with HDP 2.6.x that's why it's not available through Ambari (Install/upgrade). It has to go through our QE to be added in further release.

View solution in original post

Highlighted

Re: Adding Spark 2.2

New Contributor

Thanks Abdelkrim, do you know about any future timeline regarding the support of 2.2?

Highlighted

Re: Adding Spark 2.2

Hi @Aviram Voda

Good news for your. Spark 2.2 is supported in HDP 2.6.3 annonced today : https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.3/bk_release-notes/content/comp_versions.html

Re: Adding Spark 2.2

Contributor
@Abdelkrim Hadjidj

I tried this new version and when I load spark app using Yarn Cluster, I see the that the spark version is still: 2.1.1.2.6.2.0-205 and not 2.2...
What am I doing wrong?

Highlighted

Re: Adding Spark 2.2

New Contributor

Hi @Abdelkrim Hadjidj

Is there any future timeline for support of Spark 2.3? The new structured streaming is critical to some of our features.

Don't have an account?
Coming from Hortonworks? Activate your account here