Product Announcements

Find the latest product announcements and version updates
Celebrating as our community reaches 100,000 members! Thank you!

[ANNOUNCE] CDS 3.1 Powered by Apache Spark 3.1.1 is now GA!



CDS 3.1 powered by Apache Spark 3.1.1 is now generally available for CDP Private Cloud Base 7.1.6.

This is a minor release of CDS 3. The main improvements include:


  • The parcel contains spark3 compliant spark-hbase connector

  • All performance enhancement of Apache Spark 3.1.1 such as new optimizer rules and improved subexpression eliminations

  • Unify create table SQL syntax

  • Shuffled hash join improvement


It can be installed as an add-on parcel on top of CDP Private Cloud Base 7.1.6. The bits can be found here and the latest documentation can be found here


For Public Cloud the same bits will be available in a pre-warmed image for Cloudera Runtime 7.2.9. That can be installed through Data Engineering Spark3 Data Hub cluster templates.



Want to become a pro Spark user? Sign up for Apache Spark Training.