Announcements

When you create an operational database with HDFS storage type, the COD clusters now use 16 core instances on AWS, Azure, and GCP environments for worker nodes. The COD clusters with HDFS storage type are upgraded to enhance the performance and usability of the COD.The new worker instances for HDFS storage type are as follows:AWS: m5.4xlargeAzure: Standard_D16_v3GCP: n2-standard-16This is available starting in COD v1.41 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

We're pleased to announce the release of Cloudera Data Platform (CDP) Private Cloud Base 7.1.7 Service Pack 3 (SP3), bolstering this Long Term Supported (LTS) version of the platform further. This service pack strengthens the platform's reliability and security posture.Highlights of SP3 include:Enhanced data security and InfoSec compliance with 100+ resolved critical Customer Vulnerabilities and Exposures (CVEs) and fosters customer confidence while safeguarding their data assets. CVEs have been resolved in some of the prominent frameworks including: Netty/Jetty, Jackson databind, Guava, Apache Tomcat and Apache Shiro, Snappy Java, and Spring framework.Enhanced business continuity and data integrity by prioritizing stability and reliability to better protect the operation and integrity of your critical data, as well as other improvements such as new properties enhance retry mechanism for Kafka / Kafka Connect Ranger plugins for better handling of comm

Here are some highlights from the month of March Check out the FY24 Cloudera Meetup Events Calendar for upcoming & past event details! 133 new support questions 7 new community articles 400 new members   Rank  Community Article Author Components/ Labels #1 How to connect to CDW Iceberg table in Snowflake Zhen Zeng @zzeng  Apache Iceberg Cloudera Data Warehouse (CDW) #2 🤗 Hugging Face Spaces AMPs Accelerate ML Projects Oleksandr Akulov @aakulov  Cloudera Data Platform (CDP) Cloudera Machine Learning (CML) #3 Accelerating Replication and Decommissioning in HDFS Without Namenode Restart Srinivasu Majeti @Majeti  Cloudera Data Platform (CDP) HDFS #4 NiFi: How to detect updates to S3 files and insert the updated files into Aurora PostgreSQL Zhen Zeng @zzeng  Apache NiFi #5 CML Model Deployment with MLFlow and APIv2 Paul De Fusco @pauldefusco  Cloudera Machine Learning (CML)

Here are some highlights from the month of December Coming next on February 13th: Airflow in Action: Exploring Use Cases Across Companies Coming next on February 28th: Building Realtime AI Applications with Apache Flink Check out the FY24 Cloudera Meetup Events Calendar for upcoming & past event details! 85 new support questions 5 new community articles 399 new members Rank  Community Article Author Components/ Labels #1 Spark Python Supportability Matrix                 Ranga Reddy @RangaReddy  Apache SparkCloudera Data Engineering (CDE) #2 Working with CDE Spark Job Parameters in Cloudera Data Engineering  Paul de Fusco@pauldefusco  Apache SparkCloudera Data Engineering (CDE)) #3 Performance Delays in Namenode Caused by Multiple Intensive Operations                Karthik Palanisamy@kp

We’re excited to share the latest runtime version of Cloudera on public cloud: 7.2.18! This update is now GA, and it brings enhanced security, efficiency, and data management capabilities to your cloud environment, ensuring your infrastructure is fully supported and optimized for future innovation.What’s New?RHEL 8 is the new default OS, bringing a secure, high-performance, and cloud-optimized foundation for your big data workloadsRolling Upgrades, now GA for Data Lakes and Cloudera Operational Database, allow OS and runtime upgrades without any application downtimeApache Iceberg + Apache Atlas Integration is now GA for metadata and lineage support for governance of Iceberg tablesNew Flow Management Data Hub Tech Preview templates, based on Apache NiFi 2.0, enable you to build GenAI pipelines easier and faster using the latest version of Apache NiFiCloudera Streaming Analytics 1.12 is now available on the Streaming Analytics DataHub, with Apache Flink 1.18, Flink Application Mode, PyFl

When you create an operational database using the COD UI, the new Create Database option provides enhancements to the database creation workflow. In addition to the enhanced look and feel for better usability, the workflow also provides two additional steps called, Settings and Review.In the Settings step, you can view the default settings for your database and can also modify them. In the Review step, you can view, and finalize your changes, and also obtain a complete CLI command, in case you want to create the database using CDP CLI. Documentation link: Creating a database using CODThis is available starting in COD v1.40 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

Here are some highlights from the month of February Coming next on April 17th: Learn how to build and power enterprise AI — from data to business impact. Coming next on March 28th: Discover Data Delights: A Slice of Real-Time Analytics and GenAI! Check out the FY24 Cloudera Meetup Events Calendar for upcoming & past event details!   372 new support questions 4 new community articles 410 new members Rank  Community Article Author Components/ Labels #1 Spark and Java versions Supportability Matrix Ranga Reddy@RangaReddy  Apache SparkCloudera Data Engineering (CDE)Cloudera Data Platform (CDP) #2 How to integrated DataFlint in CDP Itay Segev@22itay  Apache Spark #3 Cloud Storage File System Operations with the Hadoop API in CDE Paul De Fusco@pauldefusco  Apache SparkCloudera Data Engineering (CDE)Cloudera Data Platform (CDP) #4 Spark Scala Version Compatibility Matrix  Ranga Reddy@Ranga

Hello CDH Users,We are pleased to announce the release of Cloudera JDBC Connector for Apache Hive 2.6.25. This release has the following fixes and enhancements:Released 2024-02-05Resolved IssuesThe following issue has been resolved in Cloudera JDBC Connector for Apache Hive 2.6.25.  * [HIVEJ-790] When connecting to a CDW cluster using HTTP transport, the    connector returns an error.Workflow Changes ============================================================= The following changes may disrupt established workflows for the connector. 2.6.23 -----------------------------------------------------------------------  * [HIVEJ-761] Removed support for Java 7    Beginning with this release, the connector no longer supports Java 7.For a    list of supported JDBC versions, see the Installation and Configuration    Guide.   * [HIVEJ-649] Removed support for Hive Serv

We are thrilled to announce the latest release of Cloudera Data Engineering for the Public Cloud. This release marks a significant milestone towards achieving parity with Spark-on-Base & Datahub. The highlight of this release is GA of In-place Upgrade, with auto backup, providing a streamlined experience for customers to transition to the latest version of CDE. Interactive Session, featuring both CLI and UI-based development, is also now GA. We are officially introducing the 2-tier pricing - CDE Core and CDE All-purpose. CDE Core virtual clusters represent our current functionality that is designed for long-running, operational pipelines. CDE All-purpose cluster includes a broader set of features compared to CDE Core including Interactive Sessions. This gives customers greater flexibility to manage costs by tailoring workloads to specific compute tiers.  Key features for this releaseHomepage 2.0 Complete revamp of the land

We are thrilled to announce the latest release of Cloudera Data Engineering for the Public Cloud. This release marks a significant milestone towards achieving parity with Spark-on-Base & Datahub. The highlight of this release is GA of In-place Upgrade, with auto backup, providing a streamlined experience for customers to transition to the latest version of CDE. Interactive Session, featuring both CLI and UI-based development, is also now GA. We are officially introducing the 2-tier pricing - CDE Core and CDE All-purpose. CDE Core virtual clusters represent our current functionality that is designed for long-running, operational pipelines. CDE All-purpose cluster includes a broader set of features compared to CDE Core including Interactive Sessions. This gives customers greater flexibility to manage costs by tailoring workloads to specific compute tiers. Key features for this releaseHomepage 2.0 Complete revamp of the landing page with a

Hello CDH Users, We are pleased to announce the release of the New Cloudera ODBC Connector 2.7.1 for Impala. This release has the following fixes and enhancements: Enhancements & New Features  [IMP-965] Multi-socket fetch support:   You can now retrieve multiple row sets from the server concurrently using multiple client connections. To do this, set EnableMultiSocketFetch to 1 [IMP-963] Improved fetch performance: The connector now has enhanced fetch performance [IMP-976] Updated third-party library: The connector now uses libcURL 8.4.0. Previously, the connector used version 8.1.2 Resolved Issues The following issue has been resolved in Cloudera ODBC Connector for Apache [IMP-974] When authenticating with SAML and an Impala CDW cluster is configured to Active-Active HA mode, the connector returns an error  Known Issues The following are known issues that you may encounter due to limitations in the data source, the connector, or an a

Here are some highlights from the month of January     Coming next on February 21st: Stay Ahead of Cyber Threats by Utilizing Data-in-Motion     Coming next on February 28th: Building Realtime AI Applications with Apache Flink Check out the FY24 Cloudera Meetup Events Calendar for upcoming & past event details! 112 new support questions 2 new community articles 462 new members Rank  Community Article Author Components/ Labels #1 How to set up CI-CD workflows in Cloudera Machine Learning to deploy your machine learning workloads   @VishRajagopalan  Vish Rajagopalan Cloudera Data Science and EngineeringCloudera Machine Learning (CML) #2 Cloudera Operational Database (COD) key features released during 2023 last quarter   @Biplab14  Biplab Chakraborty Apache HBaseApache PhoenixCloudera Operational DB We would like to recognize the below community members and employees for their ef

The latest release (2.7.0-b190) of Cloudera DataFlow (CDF) on Cloudera on cloud introduces new Change Data Capture processors, flow version tagging, deployment configuration reuse, NiFi bulletin monitoring and supports new Kubernetes versions. Key Deployment & Platform features for this releaseLatest NiFi versionFlow Deployments and Test Sessions now support the latest Apache NiFi 1.24 release.New Debezium CDC processorsCustomers can now build and deploy flows with Debezium based CDC processors for MySQL, Postgres, Oracle, SQLServer and DB2 databases.Deployment alerts for NiFi bulletinsCDF now detects bulletin error messages in NiFi flow deployments and displays them as a warning in the dashboard.Flow Definition versions can now be taggedCustomers can now use tags to identify versions easier. Common use cases for tags are applying a custom versioning scheme, or labeling flow versions as ‘ready for deployment’. The CLI introduces new commands to search flow definitions for tags

Here are some highlights from the month of November 120 new support questions 13 new community articles 474 new members Rank  Community Article Author Components/ Labels #1 Metadata in Cloudera data warehouses Anshul Gupta@Anshul_Gupta  Apache AtlasApache HiveApache Impala #2 How to optimize IMPALA/KUDU queries Shantanu Gope@ShantanuGope  Apache ImpalaApache Kudu Cloudera Data Platform (CDP) #3 How to connect to CDW (Impala) to return actively running query progress Ryan Cicak@RyanCicak  Cloudera Data Warehouse (CDW)Cloudera Machine Learning (CML) #4 CML Runtime with Nvidia Libs and VSCode Editor Paul De Fusco@pauldefusco  Cloudera Machine Learning (CML) #5 How to find Meta RegionServer detail via command line Akash Singh@aps  Apache HBase We would like to recognize the below community members and employees for their efforts over the last month to provide community solutions. 

When you create a new operational database using CDP CLI, you can enable fast autoscaling by defining the required parameters using the --auto-scaling-parameters option. COD now supports a new instance group called, Compute. The nodes under this instance group are automatically scaled up or scaled down based on the CPU utilization and RPC latency.To use fast autoscaling, you must have the COD_USE_COMPUTE_ONLY_NODES entitlement.Following is a sample command.cdp opdb create-database —environment-name <env_name> –database-name <db_name> --auto-scaling-parameters '{"minComputeNodesForDatabase":<min_compute_nodes>, "maxComputeNodesForDatabase": <max_compute_nodes>}'Documentation link: Fast autoscaling in CODThis is available starting in COD v1.37 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

When you create a new operational database with Cloud With Ephemeral Storage as the storage type on an AWS environment, COD creates the database with an I4i instance type for the worker nodes.This is available starting in COD v1.37 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

COD ensures high availability and fault tolerance using Multi-AZ deployments. A Multi-AZ deployment means that compute infrastructure for HBase’s master and region servers are distributed across multiple AZs ensuring that when a single availability zone has an outage, only a portion of Region Servers is impacted and clients automatically switch over to the remaining servers in the available AZs.Multi-AZ for COD is now supported on Microsoft Azure environments as a technical preview and is considered under development.Documentation link: Multi-AZ deployment on CODThis is available starting in COD v1.37 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

COD has removed the OPDB_USE_EPHEMERAL_STORAGE entitlement because it is not needed anymore. The use of COD on a cloud storage with ephemeral cache is enabled without an entitlement depending on the cluster creation parameters.This is available starting in COD v1.37 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

COD has removed the COD_STOREFILE_TRACKING entitlement because it is not needed anymore. The Store File Tracking (SFT) functionality is enabled on all new COD clusters created with cloud storage.This is available starting in COD v1.37 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

COD has removed the COD_EDGE_NODE entitlement now because it is not needed anymore. COD edge node functionality is enabled for all COD customers now.This is available starting in COD v1.37 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

You can deploy COD on a Google Cloud Platform (GCP) by using Google Cloud Storage (GCS) similar to what is available for Amazon Web Services (AWS) S3 storage and Microsoft Azure blob storage. Now, COD has enabled the COD_ON_GCS entitlement, by default, for such a deployment.This is available starting in COD v1.37 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

Here are some highlights from the month of October 100 new support questions 9 new community articles 476 new members Rank  Community Article Author Components/ Labels #1 Enterprise Data Quality at Scale with Spark and Great Expectations in Cloudera Data Engineering (CDE) Paul de Fusco(@pauldefusco ) Apache Airflow Apache Spark Cloudera Data Engineering (CDE) Cloudera Data Platform (CDP) #2 Spark Geospatial with Apache Sedona in Cloudera Data Engineering (CDE) Paul de Fusco(@pauldefusco ) Apache SparkCloudera Data Engineering (CDE) Cloudera Data Platform (CDP) #3 Processing real-time unstructured data with GenAI using Cloudera and IBM watsonx.ai Anshul Gupta(@Anshul_Gupta ) Apache NiFi Cloudera Data Fkow (CDF) #4 Feature Releases of Apache Spark 3 minor versions Ranga Reddy (@RangaReddy ) Apache Spark #5 New method to capturing Cloudera Manager (CM 7.x+) performance profile information Dhruv Seth (

We’re excited to announce that Cloudera’s Private Cloud Data Services 1.5.2 is now generally available.  Infuse the power of cloud-native analytics into your on-premises workloads with Cloudera’s industry leading data platform. This release delivers Cloudera’s Open Data Lakehouse on Private Cloud Data Services, providing the scale, speed, and reliability you need to build trusted, enterprise AI to solve your biggest business challenges. Here are some highlights of this release: Apache Iceberg is now generally available, enabling you to implement an Open Data Lakehouse architecture to unify all your data across a hybrid cloud ecosystem.Cloudera Data Catalog is now generally available, making it easier to find the right data at the right time, an essential piece of building out a successful Data Fabric and indeed a Data Mesh.Run new LLM Applied ML Prototypes (AMPs), including parameter efficient fine tuning (PEFT) techniques to greatly decrease the computational and s

The latest release (2.6.1-b92) of Cloudera DataFlow (CDF) on Cloudera on cloud introduces fine-grained access controls for flow drafts and deployments within an environment and support for creating and managing reporting tasks through the CLI.Key Deployment & Platform features for this releaseFine-grained access control for drafts and deployments Create projects, assign drafts and deployments, add CDP users/groups to control access.Reporting Task support Monitor your deployments with NiFi’s out of the box or custom Reporting Tasks! Reporting Tasks can now be added to deployments using new CLI commands cdp dfworkload create-reporting-task Key Flow Designer features for this releaseAdditional Zoom control Users without a scroll wheel or trackpad can now zoom in/out using ‘+/-’ controls on the canvasCollapsible sections in configuration drawer Settings, Scheduling, Properties and Relationships sections can now be collapsed for more efficient flow design To learn more ab

 We are pleased to announce the release of Cloudera JDBC 2.6.23 driver for Apache Hive.This release has the following fixes and enhancements: Enhancements and New Features[HIVEJ-748] Support for JWT authentication[HIVEJ-762] Native GSS-API support[HIVEJ-765] Updated third-party library[HIVEJ-773][02840395] Updated Jackson libraries Workflow ChangesThe following changes may disrupt established workflows for the connector[HIVEJ-761] Removed support for Java 7[HIVEJ-649] Removed support for Hive Server 1.0.0 Getting Started with the Cloudera DriverRead the Cloudera Hive JDBC 2.6.23 Driver for  Apache Hive Release Notes and Installation GuideDownload the connector from the Cloudera Connectors page As always, we welcome your feedback. Please send your comments and suggestions to the user group through our community forums. You can also file bugs through our external Jira projects on issues.cloudera.org.