Announcements

With the October release of Cloudera Cloudera AI, we have enabled the ability for CDP Machine Users to operate as Service Accounts in an ML Workspace. This feature is available both from within the ML Workspace UI and from CML APIs.The Service Account plays a crucial role in moving ML/AI workloads into production for several reasons. They enable secure access to resources and services without tying these accesses to an individual's credentials. Service Accounts also can take on roles in an ML workspace enabling scenarios where a job, deployed model, or AI application can run under the identity of that Service Account. This enables MLOps and CI/CD workloads to run under the Service Account and not tied to a particular user. Furthermore, they offer traceability, as the actions performed using a service account can be audited and monitored, ensuring accountability and governance.To learn more about this release go to our documentation.

We're excited to announce the General Availability (GA) of the Cloudera Model Registry, your centralized hub for storing, managing, and deploying machine learning models and their associated metadata. This robust tool simplifies the MLOps process, enabling organizations to develop, deploy, and maintain machine learning models effortlessly in a production environment.Cloudera's Model Registry addresses the challenges of fragmentation and lack of visibility in MLOps workflows. Serving as the single source of truth for model versions and lineage, it streamlines workflows, enhancing the traceability and reproducibility of model development.The Model Registry is now officially accessible in CML Public Cloud. To harness the full potential of General Availability (GA), upgrade your CML Workspaces and deploy your new Model Registry!You can find more information about the new Model Registry in our documentation.

Here are some highlights from the month of September Coming next on November 2nd: ICEBERG FORUM @EVOLVE Check out the FY24 Cloudera Meetup Events Calendar for upcoming & past event details! Coming next on October 26th: Virtual Event: The Open Data Lakehouse For Trusted Ai   114 new support questions 6 new community articles 3990new members Rank  Community Article Author Components/ Labels #1 Garbage Collection Pauses in Namenode and Datanode Karthik Palanisamy@kpalanisamy  Cloudera Data Platform (CDP)HDFS #2 PandasOnSpark in Cloudera Machine Learning (CML) Paul de Fusco@pauldefusco  Apache SparkCloudera Machine Learning (CML) #3 How to use Flow Definition to update flows instead of Templates Usman Khan@usman35  Apache NiFiCloudera Data Fkow (CDF) #4 How to Connect to Hiveserver2 Using Cloudera JDBC driver Asish Kumar@asish  Apache Hive #5 New Cloudera AMP with Amazon Bedrock I

 We are pleased to announce the release of Cloudera ODBC Connector 2.7.0 for Impala. This release has the following enhancements and fixes: Enhancements and New Features[IMP-940][00420456] Support for JWT authentication[IMP-941] Updated Windows Server support[IMP-951] Updated third-party libraries[IMP-958] Updated macOS support[IMP-960] [03449597] Updated Windows platform support Issues Fixed[IMP-946][02795738] The connector does not generate the last COALESCE parameter from the ELSE expression in the CASE statement  Known Issues[IMP-843][00294390] Limited support for query translation Workflow Changes[IMP-949] Updated SAML SSO cookie cache option[IMP-958] Removed support for macOS universal bitness Getting Started with the Cloudera DriverRead the Cloudera ODBC Driver 2.7.0 for Impala Release Notes and Installation Guide Download the connector from the Cloudera Connectors Page As always, we welcome your feedback. Please send your c

COD has stopped supporting the Cloudera runtime versions CDP Runtime 7.2.12 and earlier because they have reached the end of life.This is available starting in COD v1.35 onwards.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

In CDP CLI, a new option --storage-type in the create-database command is introduced that replaces the [--use-hdfs | --no-use-hdfs] and [--disable-ephemeral-storage | --no-disable-ephemeral-storage] options.The --storage-type option supports these values, CLOUD_WITH_EPHEMERAL, CLOUD, and HDFS.Following is a sample command.cdp opdb create-database --environment <environment-name> --database <database-name> --storage-type CLOUDDocumentation link: CDP CLI BetaThis is available starting in COD v1.35 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

COD using HDFS and Object Store without Ephemeral Storage, now supports upgrading the operating system version of the database using the rolling restart mode. This ensures continuous service availability during an upgrade operation.In CDP CLI, you need to use the --os-upgrade-only option along with the --rolling-upgrade option in the upgrade-database command.Following is a sample command.cdp opdb upgrade-database --environment <environment-name> --database <database-name> [--runtime <runtime-version> | --image <imageId>] --os-upgrade-only --rolling-upgradeDocumentation link: Performing a Cloudera operating system upgradeThis is available starting in COD v1.35 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

The latest release (2.6.0-b311) of Cloudera DataFlow (CDF) on Cloudera on cloud introduces NiFi 1.23, vertical scaling, several cost optimization features, developer productivity improvements and the ability to collect Diagnostic Bundles through the UI. Key Deployment & Platform features for this release Latest NiFi version: Flow Deployments and Test Sessions now support the latest Apache NiFi 1.23 release. Up to 10% savings on compute resources by removing Zookeeper from deployments and test sessions. CDF now uses a NiFi and Kubernetes native clustering protocol. Up to 50% savings on storage cost (AWS only) by moving from GP2 to GP3 volumes and introducing a new, cost efficient ‘standard’ storage profile New ‘Suspend Deployment’ behavior now terminates associated compute resources resulting in Cloudera and infrastructure cost savings. Manual Vertical scaling allows customers to change the deployment sizes easily after a deployment has been created. Allows customers to start small

We’re excited to announce the next release of Cloudera for private cloud (Private Cloud Base 7.1.9), jam packed with features and innovation to revolutionize your on-premises data experience.  This release delivers Cloudera’s Open Data Lakehouse to the data center, providing the scale, speed, and reliability you need to build trusted, enterprise AI to solve your biggest business challenges. We’re also thrilled to announce that this release will also be our next long term supported (LTS) version of the platform, delivering the stability and peace of mind that you need.Here are some highlights of this release: This LTS release offers the highest standards of enterprise readiness with various high-availability enhancements including zero downtime upgrades (ZDU), and security enhancements such as TLS 1.2.Apache Iceberg is now generally available for private cloud, enabling you to implement an Open Data Lakehouse architecture to unify all your data across a hybrid cloud ecosy

Here are some highlights from the month of July Coming next on October 25th: Storage User Group Meetup Check out the FY24 Cloudera Meetup Events Calendar for upcoming & past event details! Coming next on October 26th: Virtual Event: The Open Data Lakehouse For Trusted Ai   167 new support questions 6 new community articles 481 new members Rank  Community Article Author Components/ Labels #1 How to call a CML Deployed Model From Apache NiFi in 10 minutes Or Less Ian Brooks (ibrooks) Apache NiFi Cloudera Data Science Workbench (CDSW) Cloudera Machine Learning (CML) #2 How to integrate Apache Spark with Solr Framework Ranga Reddy (RangaReddy) Apache SolrApache SparkCloudera Data Platform (CDP)Cloudera Data Platform Private Cloud (CDP-Private) #3 How to split and transform complex XML with EvaluateXQuery Processor Samer Saleh(SAMSAL) Apache NiFi #4 Issues using newer versions of python packages such as mlflow

We are pleased to announce the general availability of Cloudera Streaming Analytics (CSA) 1.11 on CDP Private Cloud Base 7.1.9. This release includes improvements to SQL Stream Builder (SSB) as well as updates to Flink 1.16.2. These changes are focused on enhancing the user experience and fixing bugs, making the product more robust and stable.Sincere thanks to all the individuals who helped with this release and did an incredible job to get this ready.Key features for this releaseRebase to Apache Flink 1.16.2 - Apache Flink 1.16.2 is now supported in CSA 1.11.Apache Iceberg support - Support for Apache Iceberg tables using Iceberg v2 format has been added to Flink and SSB. For more information, see the Creating Iceberg tables documentation.LinksWhat's New in CSA 1.11DocumentationIceberg TablesIceberg ConnectorREST API v2 ReferenceBLOG: Building a Stateful Intrusion Detection System with SSB

COD has added a new CLI option, --java-version which can configure a major Java version on your COD cluster. The new CLI option can be used along with the create-database command to specify the Java version. The supported Java versions are JDK8 and JDK11. In case the parameter is not specified, JDK8 is used. Following is a sample command.cdp opdb create-database --environment-name <environment_name> --database-name <database_name> --java-version <value>cdp opdb create-database --environment-name cod7215 --database-name testenv --java-version 11Useful links:CDP CLI betaThis is available starting in COD v1.34 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

COD on Google Cloud Platform (GCP) can now be deployed by using Google Cloud Storage (GCS) easily, similar to what is available for Amazon Web Services (AWS) S3 storage and Microsoft Azure blob storage. The use of GCS for such a setup requires the COD_ON_GCS entitlement.COD also now supports a large ephemeral block cache while deploying on GCP. The use of ephemeral storage along with any cloud storage still requires the OPDB_USE_EPHEMERAL_STORAGE entitlement.This is available starting in COD v1.34 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

COD_ON_GCP entitlement has been removed from COD because it is not needed anymore. From this version onwards, customers can create COD clusters on Google Cloud Platform (GCP) without it.This is available starting in COD v1.34 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed)

Today, we're delighted to introduce a new and improved Community designed to enhance your Cloudera community experience. What's New in the Community? Here are some of the changes that we have worked on to enhance your engagement within the community: A Redesigned Home Page Updated Profile Settings Fresh Notifications and Message Placements, and Improved Navigation Redesigned Home Page: What to Expect The redesigned home page is now better organized. Community questions, blogs, and articles are neatly categorized into distinct sections, resulting in faster page loading times and an enhanced browsing experience. Account Information: Where to Find It To ensure a seamless experience, we've merged the Cloudera Community and MyCloudera profiles. Both accounts can now be conveniently accessed through links located in your Profile section. Notification System Changes: What's New Your profile, notifications, and private message indicators have been strategically relocated to the upper right

Here are some highlights from the month of July Coming next on September 23rd: Cloudera Now 23: THE OPEN DATA LAKEHOUSE FOR TRUSTED AI   172 new support questions 6 new community articles 501 new members Rank  Community Article Author Components/ Labels #1 How to integrate Spark3 with HBase Ranga Reddy @RangaReddy  Apache HBase Apache Spark Cloudera Data Platform (CDP) Cloudera Data Platform Private Cloud (CDP-Private) Cloudera Operational DB #2 How to integrate Spark3 with Kudu Ranga Reddy @RangaReddy  Apache Kudu Apache Spark Cloudera Data Platform (CDP) #3 CML Customize Runtime with ease (Python, R, and CUDA all in one) Ryan Cicak@RyanCicak  Cloudera Machine Learning (CML) #4 CML - Custom Runtime - VsCode using Python 3.10 Ryan Cicak@RyanCicak  Cloudera Machine Learning (CML) #5 How to change NiFi temporary staging path in case of security hardening Usman Khan@usman35&nb

You can now get a consolidated view of all the COD metrics using the Grafana dashboard. To enable the Grafana dashboard, simply use the --enable-grafana option in the create-database command. This enables the Grafana URL under the GRAFANA DASHBOARD option inside your COD database. When you click on the Grafana URL, it takes you to the Grafana dashboard.Here is a sample command:cdp opdb create-database --environment <environment_name> --database <database_name> --enable-grafana   Important links:COD documentationRelease notes This is available starting in COD v1.33 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

You can define the --scale-type for both the master and gateway nodes using CDP CLI. The --scale-type option now supports all three options [--scale-type (string) <MICRO, LIGHT, HEAVY>] for both the --master-node-type and --gateway-node-type.--scale-type LIGHT (--master-node-type LITE, --gateway-node-type LITE)--scale-type HEAVY (--master-node-type HEAVY, --gateway-node-type HEAVY)If you do not define the --scale-type, <--scale-type LIGHT> is considered for both the --master-node-type and --gateway-node-type. However, you can overwrite the --scale-type for a --gateway-node-type using the [--gateway-node-type <value>] option. Important link:CDP CLI beta This is available starting in COD v1.33 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

You can upgrade the Cloudera Runtime version of the COD database using the rolling restart mode. This functionality ensures continuous service availability during an upgrade operation. This is achieved through a new CLI parameter --rolling-upgrade | --no-rolling-upgrade in the upgrade-database command.Here is a sample command:cdp opdb upgrade-database --environment <environment-name> --database <database-name> --runtime <runtime-version> [--rolling-upgrade | --no-rolling-upgrade] Important linksCOD documentationRelease notesThis is available starting in COD v1.33 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

COD now supports faster tolling restarts. This is achieved by increasing the default value of Region Mover Threads under Cloudera Manager > HBase > Configuration to 30. This ensures a smoother rolling restart for HBase. Important links:DocumentationRelease notesThis is available starting in COD v1.33 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

 We are pleased to announce the release of Cloudera ODBC Connector 2.7.0 for Apache Hive. This release has the following enhancements and fixes: Enhancements and New Features[HAR-1272] Updated Hive server version support[HAR-1350] Updated macOS support[HAR-1344][HAR-1349] [HAR-1351] Updated third-party libraries Issues Fixed[00420434] [HAR-1337] When in ZooKeeper discovery mode and a network error occurs, the connector does not connect to the same HS2 instance on a reconnecting attempt. [HAR-1325][HAR-1330] When executing a query with CREATE TABLE AS SELECT, the connector refers to a schema as an invalid table and returns an error. Known Issues[HAR-1171] The Hive server does not currently support using scientific notation for FLOAT type.On Hive 3.0 and later, when the server does not set the metastore schema reader properly, the driver returns an error. Workflow Changes[HAR-1350] Removed support for macOS universal bitness[HAR-1346] Updated SAML SSO c

The CDP Public Cloud Release Summary for the month of July is now available. It summarizes all major features introduced in CDP Public Cloud Management Console, Data Hub, and data services. Seehttps://docs.cloudera.com/cdp-public-cloud/cloud/release-summaries/topics/announcement-202307.html 

We're thrilled to announce some exciting changes that we've been working on to improve your experience here. We can't wait to share them with all of you!   Here's a sneak peek at what's in store: A redesigned home page Updated profile and notification placement Improved navigation Home Page Redesign Our revamped page showcases community questions, blogs, and articles separated into distinct sections, resulting in faster page load times and improved browsing experience. Profile And Notification Indicators Your profile, notifications, and private message indicators have been moved to the upper right side of the page, ensuring easy access to essential account information. Additionally, we have combined the Cloudera Community and MyCloudera profiles, offering convenient access to both accounts through links available in one convenient location. Site Navigation To enhance site navigation, we've made significant updates to the community header. Now, you can easily find categories li

 We are pleased to announce the release of Cloudera JDBC Connector 2.6.32 for Impala.  This release has the following enhancements and fixes: Enhancements and New Features[IMPJ-740] Updated Jackson library: jackson-annotations 2.15.2 (previously 2.13.4)jackson-core 2.15.2 (previously 2.13.4)jackson-databind 2.15.2 (previously 2.13.4.2) Issues Fixed[02642284][IMPJ-719] On CHAR and VARCHAR types, the CAST statement fails. Getting Started with the Cloudera DriverRead the Cloudera JDBC Connector 2.6.32 for Impala Release Notes and Installation GuideDownload the connector from the Cloudera Connectors page As always, we welcome your feedback. Please send your comments and suggestions to the user group through our community forums. You can also file bugs through our external Jira projects on issues.cloudera.org.

Here are some highlights from the month of June COMING NEXT! Streaming Data Pipeline Development Check out the FY24 Cloudera Meetup Events Calendar for upcoming & past event details!   149 new support questions 34 new community articles 423 new members Rank  Community Article Author Components/ Labels #1 Spark in CML: Recommendations for using Spark in Cloudera Machine Learning Paul de Fusco  @pauldefusco  Apache Spark Cloudera Machine Learning (CML) #2 Explaining "block missing" and "block corruption" alerts in HDFS Karthik Palanisamy @kpalanisamy  HDFS #3 NIFI SSL in Modern Versions of NiFi Steven Matison @steven-matison  Apache NiFi #4 Sizing CML Workspaces: Must-Knows for properly planning workspace resources Paul de Fusco  @pauldefusco  Cloudera Data Platform (CDP)Cloudera Machine Learning (CML) #5 Publish_Kafka_1_0 processor not stops working after upgrading to CFM