New Contributor
Posts: 2
Registered: ‎10-15-2015

Spark 1.6 Support in CDH 5.5

[ Edited ]

Hi all,


Wondering if Spark 1.6 which was just released on the 4th of January will be available for immediate use with CDH 5.5? 


Spark 1.6 introduces improved memory management capabilities that dynamically manages execution and data partitions and also SQL on files.


Hoping that Cloudera is so involved in the rapid evolution of Spark that they were able to carry out the required compatibility testing prior the the official release. ;-)


Thanks in advance,


Ken Medlock

Cloudera Employee
Posts: 366
Registered: ‎07-29-2013

Re: Spark 1.6 Support in CDH 5.5

"Yes" in the sense that you should be able to run your own copy of
Spark 1.6 as a YARN app on 5.5 without much of any problem. "No" in
the sense that a minor release can't go into a maintenance release;
CDH 5.5.x == Spark 1.5.x. A future CDH minor release would be able to
take a Spark minor release.