Skip to main content
Manjunath
Manjunath
Silver ⭐️⭐️⭐️
February 7, 2019
Solved

Which version of HDP supports spark 2.4

HI All,

can somebody let me know which version of HDP supports Spark 2.4? and is that good idea to go to Spark 2.4 instead of 2.3?

My Current versions:

HDP 2.6.3.0

Apache Ambari 2.6.0.0

HDFS 2.7.3

YARN 2.7.3

MapReduce2 2.7.3

Spark2 2.2.0

Any suggestions are welcome.

Regards,

Manjunath P N

Shelton
SheltonAnswer
Gold ⭐️⭐️⭐️
February 11, 2019

@Manjunath P N

The latest HDP 3.1, unfortunately, supports spark 2.3, so you will have to wait for the next major release but after the Cloudera & Hortonworks merger, my best guess is don't expect any new HDP version anytime before the release of the combined new offering Cloudera Data Platform (CDP) sometime in 2020 or thereafter.

103446-hdp3x-spark.png

I would imagine currently HWX and CLDR should be more focused on the integration of the new products than really trying to release a newer version.

The new combined offering CDP will be based on HDP 3.x and CDH 5

HTH

View in thread

4 replies

Shelton
SheltonAnswer
Gold ⭐️⭐️⭐️
February 11, 2019

@Manjunath P N

The latest HDP 3.1, unfortunately, supports spark 2.3, so you will have to wait for the next major release but after the Cloudera & Hortonworks merger, my best guess is don't expect any new HDP version anytime before the release of the combined new offering Cloudera Data Platform (CDP) sometime in 2020 or thereafter.

103446-hdp3x-spark.png

I would imagine currently HWX and CLDR should be more focused on the integration of the new products than really trying to release a newer version.

The new combined offering CDP will be based on HDP 3.x and CDH 5

HTH

Translate
    Manjunath
    ManjunathAuthor
    Silver ⭐️⭐️⭐️
    February 11, 2019

    @Geoffrey Shelton Okot,

    Could you please share the link for screenshot which you have attached.

    Regards,

    Manjunath P N

    Translate
    Shelton
    Shelton
    Gold ⭐️⭐️⭐️
    February 11, 2019

    @Manjunath P N

    A google search should give you that HDP 3.0

    Translate
    AKR
    Gold ⭐️
    AKR
    Gold ⭐️
    December 10, 2019

    Hi,

     

    If you are looking for 2.4 on HDP alone means, as mentioned in previous emails, don't expect any new HDP version anytime before the release of the combined new offering Cloudera Data Platform (CDP) sometime in 2020 or thereafter.  As an alternative CDH 6.2+ supports spark 2.4 version.

     

    Thanks

    AK

    Translate
    Terms & ConditionsAccessibility statement