Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Which version of HDP supports spark 2.4

Solved Go to solution
Highlighted

Which version of HDP supports spark 2.4

Explorer

HI All,

can somebody let me know which version of HDP supports Spark 2.4? and is that good idea to go to Spark 2.4 instead of 2.3?

My Current versions:

HDP 2.6.3.0

Apache Ambari 2.6.0.0

HDFS 2.7.3

YARN 2.7.3

MapReduce2 2.7.3

Spark2 2.2.0

Any suggestions are welcome.

Regards,

Manjunath P N

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Which version of HDP supports spark 2.4

Mentor

@Manjunath P N

The latest HDP 3.1, unfortunately, supports spark 2.3, so you will have to wait for the next major release but after the Cloudera & Hortonworks merger, my best guess is don't expect any new HDP version anytime before the release of the combined new offering Cloudera Data Platform (CDP) sometime in 2020 or thereafter.

103446-hdp3x-spark.png

I would imagine currently HWX and CLDR should be more focused on the integration of the new products than really trying to release a newer version.

The new combined offering CDP will be based on HDP 3.x and CDH 5

HTH

View solution in original post

4 REPLIES 4
Highlighted

Re: Which version of HDP supports spark 2.4

Mentor

@Manjunath P N

The latest HDP 3.1, unfortunately, supports spark 2.3, so you will have to wait for the next major release but after the Cloudera & Hortonworks merger, my best guess is don't expect any new HDP version anytime before the release of the combined new offering Cloudera Data Platform (CDP) sometime in 2020 or thereafter.

103446-hdp3x-spark.png

I would imagine currently HWX and CLDR should be more focused on the integration of the new products than really trying to release a newer version.

The new combined offering CDP will be based on HDP 3.x and CDH 5

HTH

View solution in original post

Highlighted

Re: Which version of HDP supports spark 2.4

Explorer

@Geoffrey Shelton Okot,

Could you please share the link for screenshot which you have attached.

Regards,

Manjunath P N

Highlighted

Re: Which version of HDP supports spark 2.4

Mentor

@Manjunath P N

A google search should give you that HDP 3.0

Highlighted

Re: Which version of HDP supports spark 2.4

Cloudera Employee

Hi,

 

If you are looking for 2.4 on HDP alone means, as mentioned in previous emails, don't expect any new HDP version anytime before the release of the combined new offering Cloudera Data Platform (CDP) sometime in 2020 or thereafter.  As an alternative CDH 6.2+ supports spark 2.4 version.

 

Thanks

AK

Don't have an account?
Coming from Hortonworks? Activate your account here