We are planning to Upgrade our HDP from 2.4.2 to 2.6.2 because we need to use the Spark 2.2.0 in our project.
We are asked to use the Spark version 2.2.0 from our security team as it is latest vulnerable free version.
So the question here is,
Is Spark 2.1.1 which comes with HDP package 2.6.2 is vulnerable free ? because as we read and understand the latest vulnerable free version of Apache Spark available is 2.2.0 .
Thanks in Advance ,
Hi @Param NC
Can you be more specific about vulnerable free? do you have a Bug or Jira ID for the issue you are referring to ?
FYI, HDP 2.6.3 available since 2 days has Spark 2.2 GA so you can use it.
Thanks for the quick response ,
One such issue I can provide is https://issues.apache.org/jira/browse/SPARK-20922 which affected the Spark 2.1.1 .
and few I got from the link https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=spark , where it clearly says to use the Spark version 2.2.0 in many places .
Thanks for details @Param NC
SPARK-20922 is already corrected in HDP 2.6.2 as you can see here : https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_release-notes/content/patch_spark.html
You can check this list also for any other issue.
As I said, Spark 2.2 is GA if you can use HDP 2.6.3