Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Spark 1.6 Support in CDH 5.5

Highlighted

Spark 1.6 Support in CDH 5.5

New Contributor

Hi all,

 

Wondering if Spark 1.6 which was just released on the 4th of January will be available for immediate use with CDH 5.5? 

  

Spark 1.6 introduces improved memory management capabilities that dynamically manages execution and data partitions and also SQL on files.

 

Hoping that Cloudera is so involved in the rapid evolution of Spark that they were able to carry out the required compatibility testing prior the the official release. ;-)

 

Thanks in advance,

 

Ken Medlock

1 REPLY 1

Re: Spark 1.6 Support in CDH 5.5

Master Collaborator
"Yes" in the sense that you should be able to run your own copy of
Spark 1.6 as a YARN app on 5.5 without much of any problem. "No" in
the sense that a minor release can't go into a maintenance release;
CDH 5.5.x == Spark 1.5.x. A future CDH minor release would be able to
take a Spark minor release.