What's New @ Cloudera

Find the latest Cloudera product news
Celebrating as our community reaches 100,000 members! Thank you!

Cloudera Data Engineering supports In-place Upgrade, Interactive Sessions, Git Integration and Airflow Custom Operators in the latest Public Cloud 1.20.3 release

Cloudera Employee

We are thrilled to announce the latest release of Cloudera Data Engineering for the Public Cloud. 

This release marks a significant milestone towards achieving parity with Spark-on-Base & Datahub. The highlight of this release is GA of In-place Upgrade, with auto backup, providing a streamlined experience for customers to transition to the latest version of CDE. 

Interactive Session, featuring both CLI and UI-based development, is also now GA. We are officially introducing the 2-tier pricing - CDE Core and CDE All-purposeCDE Core virtual clusters represent our current functionality that is designed for long-running, operational pipelines. CDE All-purpose cluster includes a broader set of features compared to CDE Core including Interactive Sessions. This gives customers greater flexibility to manage costs by tailoring workloads to specific compute tiers. 



Key features for this release

  • Homepage 2.0 

    • Complete revamp of the landing page with a new design system

    • Focus on develop, deploy, monitor workflow



  • In-place Upgrade GA

    • Single-click upgrades driven completely via a wizard in the UI

    • Added backup and restore functionality for any failure scenarios



  • Git Integration Tech Preview(Demo videos)

    • Integration with leading Git providers such as GitHub and Bitbucket

    • Added support for Git files in CDE Spark and Airflow jobs

  • Interactive Sessions GA

    • A flexible, interactive development interface

    • Added functionality to attach resources with Sessions

    • Support Pyspark, Scala and Java

  • Airflow Custom Operators(Demo videos)

    •  Support for third-party Operators/Libraries to build custom pipelines

  • Spark Streaming Tech Preview

  • Group & Role-Based VC Access Control

    • Implemented access controls for both groups and users, offering admin or read-only permissions

  • Add more AWS & Azure Instances

  • K8s 1.27 Support (Learn more)

  • Airflow Upgrade from 2.3 to 2.6

Additional resources to help: