We are thrilled to announce the latest release of Cloudera Data Engineering for the Public Cloud.
This release marks a significant milestone towards achieving parity with Spark-on-Base & Datahub. The highlight of this release is GA of In-place Upgrade, with auto backup, providing a streamlined experience for customers to transition to the latest version of CDE.
Interactive Session, featuring both CLI and UI-based development, is also now GA. We are officially introducing the 2-tier pricing - CDE Core and CDE All-purpose. CDE Core virtual clusters represent our current functionality that is designed for long-running, operational pipelines. CDE All-purpose cluster includes a broader set of features compared to CDE Core including Interactive Sessions. This gives customers greater flexibility to manage costs by tailoring workloads to specific compute tiers.
Key features for this release
Homepage 2.0
Complete revamp of the landing page with a new design system
Focus on develop, deploy, monitor workflow
In-place Upgrade GA
Single-click upgrades driven completely via a wizard in the UI
Added backup and restore functionality for any failure scenarios
Git Integration Tech Preview(Demo videos)
Integration with leading Git providers such as GitHub and Bitbucket
Added support for Git files in CDE Spark and Airflow jobs
Interactive Sessions GA
A flexible, interactive development interface
Added functionality to attach resources with Sessions
Support Pyspark, Scala and Java
Airflow Custom Operators(Demo videos)
Support for third-party Operators/Libraries to build custom pipelines
Spark Streaming Tech Preview
Group & Role-Based VC Access Control
Implemented access controls for both groups and users, offering admin or read-only permissions
Add more AWS & Azure Instances
K8s 1.27 Support (Learn more)
Airflow Upgrade from 2.3 to 2.6
Additional resources to help:
