Announcements

Here are some highlights from the month of August     WEBINAR Introducing AI in a Box for Key Industries Watch Now     VIRTUAL EVENT The latest innovations in data, analytics & AI October 15, 2025 8:00 AM PT | 11:00 AM ET | 4:00 PM GMT | 5:00 PM CES Register Now Check out the FY25 Cloudera Meetup Events Calendar for upcoming & past event details!   1160 members 1 new articles 47 questions We would like to recognize the below community members and employees for their efforts over the last month to provide community solutions.  See all our top participants at Top Solution Authors leaderboard and all the other leaderboards on our Leaderboards and Badges page.     @MattWho @Scharan @jagadeesan @vafs @yagoaparecidoti @PradNiFi1236 @Former Member @Dalier  Share your expertise and answer some of the below open questions. Also, be sure to bookmark the unanswered qu

We are excited to announce the release of Cloudera Distribution of Apache Spark (CDS) 3.5 for Cloudera on premises 7.1.9. This new release is available as an add-on parcel and provides long-term support for Apache Spark 3.5, the latest Long-Term Support (LTS) version adopted by both Cloudera and the broader Apache community.What's Changing?This update is for customers using the CDS 3.3 add-on parcel to use Spark 3 with Cloudera on premises 7.1.9.  Key Benefits of Upgrading to Apache Spark 3.5Cloudera recommends upgrading to CDS 3.5 for all users currently running CDS 3.3. This upgrade offers significant advantages:Performance Boost: Spark 3.5 is expected to deliver a performance increase of over 20% compared to Spark 3.3. This means faster data processing and more efficient workloads for your enterprise.Enhanced Features: This release includes enhanced security, bug fixes, and support for newer versions of Python. These improvements ensure your data engineering workflows are

Here are some highlights from the month of August     WEBINAR Introducing AI in a Box for Key Industries Register Now   VIRTUAL EVENT The latest innovations in data, analytics & AI October 15, 2025 8:00 AM PT | 11:00 AM ET | 4:00 PM GMT | 5:00 PM CEST   Register Now     Check out the FY25 Cloudera Meetup Events Calendar for upcoming & past event details!   1360 members 5 new articles 70 questions We would like to recognize the below community members and employees for their efforts over the last month to provide community solutions.  See all our top participants at Top Solution Authors leaderboard and all the other leaderboards on our Leaderboards and Badges page.     @upadhyayk04 @MattWho @Gopinath @jagadeesan @rsanchez @RamaClouder @mslnrd @Alf015 @SAMSAL @Former Member  Share your expertise and answer some of the below open questions.

Here are some highlights from the month of July     WEBINARThe Power of Streaming in Real-Time AI and AnalyticsRegister Now   VIRTUAL EVENTThe latest innovations in data, analytics & AIWatch Now On Demand Check out the FY25 Cloudera Meetup Events Calendar for upcoming & past event details!   60 New support questions 1095 New members We would like to recognize the below community members and employees for their efforts over the last month to provide community solutions.  See all our top participants at Top Solution Authors leaderboard and all the other leaderboards on our Leaderboards and Badges page.   @MattWho @vats @mburgess @upadhyayk04 @Boris G @yagoaparecidoti @BobKing  Share your expertise and answer some of the below open questions. Also, be sure to bookmark the unanswered question page to find additional open questions.  Unanswered Community Post  Compon

Cloudera’s Data In Motion Team is pleased to announce the release of the Cloudera Streaming Analytics - Kubernetes Operator 1.3, an integral component of Cloudera Streaming - Kubernetes Operators. This release includes rebases to Apache Flink 1.20 and Apache Flink Kubernetes Operator 1.11.0. Other changes and updates are focused on enhancing security, usability, and making the product more robust.Release HighlightsRebase to Flink 1.20For more information, see the Flink 1.20 Release Announcement and the Release Notes.Rebase to Flink Kubernetes Operator 1.11.0For more information, see the Apache Flink Kubernetes Operator 1.11.0 Release Announcement and the Release Notes.Flink OpenTelemetry Metrics Reporter (Technical Preview)The OpenTelemetry metrics reporter is now included, in Tech Preview, in the operator image. It makes it easier and more efficient to aggregate job metrics to a central service like Prometheus or any OpenTelemetry-compatible service, and aggregate metrics using open s

CommunityOverCode Asia 2025 will take place from July 25-27, 2025, in Beijing. CommunityOverCode (formerly ApacheCon) is the official global conference series of the Apache Software Foundation (ASF). Through keynote, case studies, training, and hackathons, the conference showcases the latest developments and innovations across Apache projects.  The Apache ecosystem's core value lies in open-source collaboration and community-driven innovation. Apache project contributors, committers, and technical experts from around the world will gather at CommunityOverCode Asia 2025 to share their latest technical insights and experiences in AI, big data, storage, analytics, and more. Cloudera's open-source contributors will present across 4 different venues with 7 technical sessions, comprehensively demonstrating how Apache technology stack applications are transforming modern data processing. From data storage to analytics, we'll share how open-source technologies solve complex data challenge

The Data In Motion team is thrilled to announce the release of Cloudera Streams Messaging - Kubernetes Operator 1.4 and the exciting new Cloudera Surveyor! This release marks a significant step forward in simplifying and enhancing your Apache Kafka management experience on Kubernetes, focusing on intuitive user experience and powerful new capabilities.Release HighlightsIntroducing Cloudera Surveyor: Your New Go-To for Seamless Kafka Management on Kubernetes Cloudera Surveyor is a brand-new, intuitive graphical interface built to streamline the monitoring and management of your Apache Kafka clusters running on Kubernetes. As a core component delivered with Cloudera Streams Messaging - Kubernetes Operator 1.4, Surveyor empowers you with:Effortless Management: Move beyond the command line with a modern, easy-to-use UI for all your Kafka operations.Multi-Cluster Visibility: Manage and monitor multiple Kafka clusters from a single, centralized console.Actionable Insights: Get real-time

Cloudera Data Engineering 1.24.1We are excited to announce the latest release of Cloudera Data Engineering on Cloud. This release introduces advancements focused on optimizing cost, enhancing security, and improving the overall developer and administrative experience.Key FeaturesSuspend and Resume (Technical Preview): Temporarily suspend your entire Cloudera Data Engineering Service during idle periods to achieve up to 80%  cost savings.Git Integration Support (GA): The Git integration feature is now generally available.  Stay tuned for more enhancements in the upcoming releases.Security Hardened Images:  As part of our ongoing effort to improve security and resilience against Common Vulnerabilities and Exposures (CVEs), new hardened images are used by default during service and virtual cluster creation.  In addition, new hardened images for Spark and Airflow will be available that contain runtime components upgrades, including Python and JDK. For more information,

Here are some highlights from the month of June ANALYST REPORTThe Future of Enterprise AI Agents Download the Report HERE VIRTUAL EVENTThe latest innovations in data, analytics & AI Register Now Check out the FY25 Cloudera Meetup Events Calendar for upcoming & past event details! 62 new support questions 1095 new members Community Article Author Components/ Labels Lighthouse FAQs - The New Support Portal (GA in June 2025)  Madhubrat Patra Cloudera Search From SAP S/4HANA to Lakehouse: A Practical Guide Using OData API and Apache NiFi Zhen Zeng Apache IcebergApache NiFi ZooKeeper removal from Cruise Control Tamas Barnabas Egyed Apache KafkaApache Zookeeper Introduction to CDE Job and Resource ACLs  Paul DeFusco Apache AirflowApache SparkCloudera Data ENgineering (CDE) We would like to recognize the below community members and employees for their efforts over the last month to provide community so

Here are some highlights from the month of April       ANALYST REPORT The Future of Enterprise AI Agents Download the Report HERE       WEBINAR Data Governance in the Age of AI Watch now HERE Check out the FY25 Cloudera Meetup Events Calendar for upcoming & past event details!     67 new support questions 4 new community articles 1115 new members Community Article Author Components/ Labels Python and Jython in Cloudera Flow Management @mburgess  Apache NiFi Cloudera DataFlow (CDF) Understanding Audits in CDP(Public Cloud): Why They Matter and How to Use Them @shehbazk  Cloudera Data Platform (CDP) Recommendations for Using Dask Cuda Clusters in Cloudera AI @pauldefusco  Cloudera Machine Learning (CML) CDE Jobs with Multiple CDE Repository Dependencies Apache Spark Cloudera Data Engineering (CDE) Ingesting 3rd Party Data into the Cloudera Data Lake through Clo

We are excited to announce long-term support for Apache Spark 3.5 in the Cloudera Unified Runtime (7.3.1 SP1). This update brings the newest Spark version and a more modern, high-performance experience for your data engineering teams. Key BenefitsUp to 50% faster long-running queries, with 5-10% overall performance gains from advanced query planning, adaptive execution, and dynamic partition pruning.Modern developer experience with wider Python support (PySpark) and broader SQL coverage.Stronger stability and scalability with improved memory management, fault tolerance, and support for large-scale workloads.Next StepsAs of Cloudera Runtime 7.3.1.200 SP1, Spark 3 is the default version, and Spark 2 has been fully removed.If you already use Spark 3, upgrading is simple – just recompile your applications for Spark 3.5 and continue running your jobs. Migrating Spark Applications has useful resources to consider when moving between versions.Migrating from Spark 2? Start with these reso

SELinux allows you to set access control through policies. You can set the SELinux mode while creating a new operational database. You can configure the SELinux enforcement for your Cloudera Operational Database using the Cloudera Operational Database UI or CDP CLI.This feature is under technical preview. To use this feature, you must have the CDP_SECURITY_ENFORCING_SELINUX entitlement in your Cloudera environment and the minimum runtime version of 7.2.18.700 or 7.3.1.100. Contact Cloudera support if you do not have this entitlement.Using the CDP CLI:You can define the SELinux mode using the seLinux parameter in the create-database command. The supported SELinux modes are:ENFORCING: Enables SELinux in enforced mode, actively applying security policies.PERMISSIVE (default): Sets SELinux to permissive mode, logging any security violations without enforcing policies.If you do not define the seLinux parameter, by default, the PERMISSIVE mode is applied.The following example shows usage of

Cloudera Operational Database cluster deployments on AWS Graviton environments are now generally available. AWS Graviton is a family of general-purpose, ARM-based processors designed for cloud workloads. Cloudera Operational Database supports the following Graviton versions: Graviton2: Used in Cloudera Operational Database deployments on I4G instances Graviton4: Used in Cloudera Operational Database deployments on I8G instances AWS Graviton processors deliver exceptional price performance for workloads running on AWS EC2. With Graviton4, you can further optimize costs while achieving superior performance. For additional details, see the AWS press release. To help you choose the most suitable ARM processor for your performance requirements, consult the AWS Graviton Processors documentation, which provides a detailed comparison between the Graviton2 and Graviton4 instances. For more information on AWS Graviton support, see AWS Graviton instances in Cloudera Operational Database. This f

The Data In Motion team is pleased to announce the release of Cloudera Streaming Analytics 1.15 for Cloudera  7.3.1. This release  focuses on enhancing the user experience and adding new important features to the product, and includes improvements to SQL Stream Builder as well as a rebase to Apache Flink 1.20.1..Release HighlightsCloudera platform supportCloudera Streaming Analytics 1.15 is supported on Cloudera 7.3.1.100 (Cumulative Hot Fix 1). Ensure that you review the 7.3.1.100 Release Notes and Support Matrix to understand which operating system, database, and JDK versions are supported for Cloudera Streaming Analytics as well.Rebase to Apache Flink 1.20Streaming analytics deployments, including SQL Stream Builder, now support Apache Flink 1.20. For more information on what is included in the Apache Flink 1.20 version, see the Apache Flink 1.20 Release Announcement and Release Notes.Support for batch-mode queries in Cloudera SQL Stream BuilderUsers can select and use “ba

Release HighlightsRebase on Flink 1.19.2For more information, see the Flink 1.19.2 Release Announcement and the Release Notes.General availability for Cloudera SQL Stream BuilderCloudera SQL Stream Builder, previously in Technical Preview, is now generally available in Cloudera Streaming Analytics - Kubernetes Operator.Cloudera SQL Stream Builder is a comprehensive interactive user interface for creating stateful stream processing jobs using SQL. For more information about SQL Stream Builder and its features, see the Getting started with SQL Stream Builder page.LDAP authentication in SQL Stream BuilderLDAP-based authentication is available for Cloudera SQL Stream Builder. For more information, refer to LDAP authentication.Custom truststoresYou can now specify custom truststores when installing the Cloudera Streaming Analytics - Kubernetes Operator. For more information, refer to Security configurations.Secure TLS connections to the SSB UI and APIUsers can connect to Cloudera SQL Stream

Cloudera’s Data In Motion Team is pleased to announce the release of Cloudera Streams Messaging - Kubernetes Operator 1.3, an integral component of Cloudera Streaming - Kubernetes Operator. With this release, customers receive a rebase to Kafka 3.9, automatic cluster rebalance, better offset management capabilities for Kafka connectors, and more!Release HighlightsRebase to Kafka 3.9For more information, see the Kafka 3.9 Release Notes and the list of notable changes.Rebase to Strimzi 0.45.0For more information, see the Strimzi 0.44.0 Release Notes and Strimzi 0.45.0 Release Notes.KRaft (Kafka Raft) is generally availableYou can now deploy Kafka clusters that use KRaft instead of ZooKeeper for metadata management. Additionally, you can migrate existing ZooKeeper-based Kafka clusters to use KRaft. With the addition of KRaft, ZooKeeper is deprecated. Deploying new or using existing Kafka clusters running in ZooKeeper mode is deprecated. Additionally, ZooKeeper will be removed in a future

Here are some highlights from the month of January       Coming on March 13th: The Power of Metadata: Build an interoperable data estate you can control and trust. Register Now! Check out the FY25 Cloudera Meetup Events Calendar for upcoming & past event details!   713 new support questions 8 new community articles 731 new members Community Article Author Components/ Labels Fully Private Agents with Cloudera's AI Inference Service and CrewAI @shresh  Cloudera Data Science and EngineeringCloudera Machine Learning (CML) Iceberg WAP – Failsafe ETL with Iceberg and CDE  @wdyson  Apache IcebergCloudera Data Engineering (CDE)Cloudera Data Warehouse (CDW) Getting Started with Spark GraphFrames in Cloudera AI Workbench @pauldefusco  Apache SparkCloudera Data Platform (CDP)Cloudera Machine Learning (CML) JupyterLab and Spark Connect Quickstart in Cloudera Data Engineering Apache SparkCloudera Da

Accessing Apache Spark from Your Favorite IDEWorking with Apache Spark just got easier. The recent release of Cloudera Data Engineering 1.23 introduces External IDE Connectivity, powered by Spark Connect. You can now interact with Spark clusters directly from your preferred local coding environments like Jupyter Notebook, VS Code, and PyCharm. This capability helps to streamline your data engineering workflows, enabling faster development and better collaboration, all while preserving enterprise-grade security. This 6-minute demo will walk you through how to access Spark from your favorite IDE with Cloudera Data Engineering:Key BenefitsExternal IDE Connectivity brings a new level of flexibility for data engineering, allowing users to work with remote data from their local environment via secure, automated continuous integration and continuous delivery (CI/CD) pipelining. This approach offers several advantages, including:1. Develop Locally, Compute SeamlesslyDevelopers can connect

Introduction As organizations increasingly migrate their data workloads to the cloud, choosing the right storage solution is crucial. This article compares Spark3 on YARN workloads using two different Amazon S3 storage classes: S3 Standard and S3 Express One Zone running Cloudera on AWS with a 7.2.18 CDP runtime version. We'll look at the results of a series of benchmark tests highlighting the performance differences, cost implications, and key considerations for using these two storage options. Background and Motivation In cloud computing, selecting the right storage solution directly impacts the overall performance, availability, and cost of your data applications. Spark workloads that use Amazon S3 storage often face choices between multiple storage classes, each offering distinct features and trade-offs. This blog provides insight into the performance of Spark3 on YARN workloads when using S3 Standard compared to S3 Express One Zone, leveraging the industry-standard benchmarking me

Cloudera Operational Database supports setting a user as an ODAdmin at the database level. In earlier versions of the Cloudera Operational Database, you could set the ODAdmin at the environment level only, however, for better usability and enhanced security, now you can set it at the database level too. This is available starting in COD v1.48 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

Cloudera Operational Database supports creating a database with SELinux enforcement using the Cloudera Operational Database CLI. To use SELinux support, you must have the CDP_SECURITY_ENFORCING_SELINUX entitlement. Please get in touch with Cloudera support if you do not have this entitlement.The SELinux allows you to set access control through policies. You can set the SELinux mode while creating a new operational database. You can define the SELinux mode using the seLinux parameter in the create-database command. The supported SELinux modes are:ENFORCING: Enables SELinux in enforced mode, actively applying security policies.PERMISSIVE (default): Sets SELinux to permissive mode, logging any security violations without enforcing policies.If you do not define the seLinux parameter, by default, the PERMISSIVE mode is applied.The following example shows the usage of the seLinux parameter. opdb create-database --environment-name [***ENVIRONMENT_NAME***] --database-name [

We are pleased to announce the release of Cloudera Streams Messaging - Kubernetes Operator 1.2. With this release, customers receive better security integration and an update to Kafka 3.8, besides other improvements.Release HighlightsRebase on Kafka 3.8:For more information, see the Kafka 3.8 Release Notes and list of notable changes.Rebase on Strimzi 0.43.0:For more information, see the Strimzi 0.43.0 Release Notes.Apache Ranger authorization: Support for Apache Ranger authorization is now available. Customers can now integrate Kafka clusters, deployed with Cloudera Streams Messaging - Kubernetes Operator, with a remote Ranger service that is running on Cloudera Private Cloud Base. If configured, the Ranger service can provide authorization for your Kafka cluster. For more information, see Apache Ranger authorization.Improvements to Kafka replication: Rebased and backported changes to make Kafka replication more resilient and reliable when handling heartbeats and offset translation.Pe

We are pleased to announce the release of Cloudera Streaming Analytics 1.14 for Cloudera Public Cloud and Cloudera Private Cloud Base 7.3.1. This release includes improvements to SQL Stream Builder as well as updates to Apache Flink 1.19.1.Release HighlightsRebase to Apache Flink 1.19.1: Streaming analytics deployments, including SQL Stream Builder, now support Apache Flink 1.19.1, which includes the Apache Flink improvements below. For more information on these improvements and deprecations, please check the Apache Flink 1.19.1 release announcement.Custom Parallelism for Table/SQL Sources: The DataGen connector now supports setting of custom parallelism for performance tuning via the scan.parallelism option. Support for other connectors will come in future releases.Configure Different State Time to Live (TTLs) Using SQL Hint: Users have now a more flexible way to specify custom time-to-live (TTL) values for state of regular joins and group aggregations directly within t

We are thrilled to share the latest release of Cloudera Data Engineering 1.23 on public cloud. With major enhancements for developer experience and product stability, this release enables you to benefit from increased productivity and collaboration while ensuring enterprise-grade security. Discover New Benefits & FeaturesBoosted Development Productivity: The External IDE Connectivity (Tech Preview) powered by Apache Spark Connect enables practitioners to run large-scale Spark workloads remotely on Cloudera Data Engineering. That means you can access Spark from your favorite IDE and collaborate more efficiently with your team. Optimized, Cost-Effective Workflows: Support for Apache Airflow 2.9 and Python 3.11 expands access to the latest libraries and frameworks, equipping you with tools to meet your dynamic needs for building scalable data pipelines. Apache Iceberg 1.5, together with the support of Apache Spark 3.5, delivers better overall performance and cost management.

The feature to scale up the HBase REST server using the Apache HBase REST API is no longer a technical preview and is now generally available (GA). For more information, see HBase REST server scaling in COD.This is available starting in COD v1.46 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).