02-06-2019 09:48 AM
I'm currently running a cluster with Cloudera Enterprise 5.16.1 and I've upgraded it to use Spark 2.2. All is working as expected.
I now realise there's a feature in Spark 2.4 I'm keen to use but can't see an easy way to update my cluster since the latest CDS version available is 2.3 (https://www.cloudera.com/documentation/spark2/latest/topics/spark2_packaging.html#versions)
Is there an alternative way to get Spark 2.4 installed on the cluster or am I missing something obvious?