Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is Spark 2.1.1 comes with HDP 2.6.2 has all the identified bugs from previous versions fixed ?

Highlighted

Is Spark 2.1.1 comes with HDP 2.6.2 has all the identified bugs from previous versions fixed ?

Contributor

Dear All,

We are planning to Upgrade our HDP from 2.4.2 to 2.6.2 because we need to use the Spark 2.2.0 in our project.

We are asked to use the Spark version 2.2.0 from our security team as it is latest vulnerable free version.

So the question here is,

Is Spark 2.1.1 which comes with HDP package 2.6.2 is vulnerable free ? because as we read and understand the latest vulnerable free version of Apache Spark available is 2.2.0 .

Thanks in Advance ,

Param.

3 REPLIES 3

Re: Is Spark 2.1.1 comes with HDP 2.6.2 has all the identified bugs from previous versions fixed ?

Hi @Param NC

Can you be more specific about vulnerable free? do you have a Bug or Jira ID for the issue you are referring to ?

FYI, HDP 2.6.3 available since 2 days has Spark 2.2 GA so you can use it.

Thanks

Re: Is Spark 2.1.1 comes with HDP 2.6.2 has all the identified bugs from previous versions fixed ?

Contributor

Thanks for the quick response ,

One such issue I can provide is https://issues.apache.org/jira/browse/SPARK-20922 which affected the Spark 2.1.1 .

and few I got from the link https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=spark , where it clearly says to use the Spark version 2.2.0 in many places .

Thanks,

Re: Is Spark 2.1.1 comes with HDP 2.6.2 has all the identified bugs from previous versions fixed ?

Thanks for details @Param NC

SPARK-20922 is already corrected in HDP 2.6.2 as you can see here : https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_release-notes/content/patch_spark.html

You can check this list also for any other issue.

As I said, Spark 2.2 is GA if you can use HDP 2.6.3