What's New @ Cloudera

 We are pleased to announce the release of Cloudera ODBC Connector 2.7.0 for Impala. This release has the following enhancements and fixes: Enhancements and New Features[IMP-940][00420456] Support for JWT authentication[IMP-941] Updated Windows Server support[IMP-951] Updated third-party libraries[IMP-958] Updated macOS support[IMP-960] [03449597] Updated Windows platform support Issues Fixed[IMP-946][02795738] The connector does not generate the last COALESCE parameter from the ELSE expression in the CASE statement  Known Issues[IMP-843][00294390] Limited support for query translation Workflow Changes[IMP-949] Updated SAML SSO cookie cache option[IMP-958] Removed support for macOS universal bitness Getting Started with the Cloudera DriverRead the Cloudera ODBC Driver 2.7.0 for Impala Release Notes and Installation Guide Download the connector from the Cloudera Connectors Page As always, we welcome your feedback. Please send your c

COD has stopped supporting the Cloudera runtime versions CDP Runtime 7.2.12 and earlier because they have reached the end of life.This is available starting in COD v1.35 onwards.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

In CDP CLI, a new option --storage-type in the create-database command is introduced that replaces the [--use-hdfs | --no-use-hdfs] and [--disable-ephemeral-storage | --no-disable-ephemeral-storage] options.The --storage-type option supports these values, CLOUD_WITH_EPHEMERAL, CLOUD, and HDFS.Following is a sample command.cdp opdb create-database --environment <environment-name> --database <database-name> --storage-type CLOUDDocumentation link: CDP CLI BetaThis is available starting in COD v1.35 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

COD using HDFS and Object Store without Ephemeral Storage, now supports upgrading the operating system version of the database using the rolling restart mode. This ensures continuous service availability during an upgrade operation.In CDP CLI, you need to use the --os-upgrade-only option along with the --rolling-upgrade option in the upgrade-database command.Following is a sample command.cdp opdb upgrade-database --environment <environment-name> --database <database-name> [--runtime <runtime-version> | --image <imageId>] --os-upgrade-only --rolling-upgradeDocumentation link: Performing a Cloudera operating system upgradeThis is available starting in COD v1.35 and you do not need to upgrade to a higher version of COD to benefit from this feature.Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

The latest release (2.6.0-b311) of Cloudera DataFlow (CDF) on Cloudera on cloud introduces NiFi 1.23, vertical scaling, several cost optimization features, developer productivity improvements and the ability to collect Diagnostic Bundles through the UI. Key Deployment & Platform features for this release Latest NiFi version: Flow Deployments and Test Sessions now support the latest Apache NiFi 1.23 release. Up to 10% savings on compute resources by removing Zookeeper from deployments and test sessions. CDF now uses a NiFi and Kubernetes native clustering protocol. Up to 50% savings on storage cost (AWS only) by moving from GP2 to GP3 volumes and introducing a new, cost efficient ‘standard’ storage profile New ‘Suspend Deployment’ behavior now terminates associated compute resources resulting in Cloudera and infrastructure cost savings. Manual Vertical scaling allows customers to change the deployment sizes easily after a deployment has been created. Allows customers to start small

We’re excited to announce the next release of Cloudera for private cloud (Private Cloud Base 7.1.9), jam packed with features and innovation to revolutionize your on-premises data experience.  This release delivers Cloudera’s Open Data Lakehouse to the data center, providing the scale, speed, and reliability you need to build trusted, enterprise AI to solve your biggest business challenges. We’re also thrilled to announce that this release will also be our next long term supported (LTS) version of the platform, delivering the stability and peace of mind that you need.Here are some highlights of this release: This LTS release offers the highest standards of enterprise readiness with various high-availability enhancements including zero downtime upgrades (ZDU), and security enhancements such as TLS 1.2.Apache Iceberg is now generally available for private cloud, enabling you to implement an Open Data Lakehouse architecture to unify all your data across a hybrid cloud ecosy

We are pleased to announce the general availability of Cloudera Streaming Analytics (CSA) 1.11 on CDP Private Cloud Base 7.1.9. This release includes improvements to SQL Stream Builder (SSB) as well as updates to Flink 1.16.2. These changes are focused on enhancing the user experience and fixing bugs, making the product more robust and stable.Sincere thanks to all the individuals who helped with this release and did an incredible job to get this ready.Key features for this releaseRebase to Apache Flink 1.16.2 - Apache Flink 1.16.2 is now supported in CSA 1.11.Apache Iceberg support - Support for Apache Iceberg tables using Iceberg v2 format has been added to Flink and SSB. For more information, see the Creating Iceberg tables documentation.LinksWhat's New in CSA 1.11DocumentationIceberg TablesIceberg ConnectorREST API v2 ReferenceBLOG: Building a Stateful Intrusion Detection System with SSB

COD has added a new CLI option, --java-version which can configure a major Java version on your COD cluster. The new CLI option can be used along with the create-database command to specify the Java version. The supported Java versions are JDK8 and JDK11. In case the parameter is not specified, JDK8 is used. Following is a sample command.cdp opdb create-database --environment-name <environment_name> --database-name <database_name> --java-version <value>cdp opdb create-database --environment-name cod7215 --database-name testenv --java-version 11Useful links:CDP CLI betaThis is available starting in COD v1.34 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

COD on Google Cloud Platform (GCP) can now be deployed by using Google Cloud Storage (GCS) easily, similar to what is available for Amazon Web Services (AWS) S3 storage and Microsoft Azure blob storage. The use of GCS for such a setup requires the COD_ON_GCS entitlement.COD also now supports a large ephemeral block cache while deploying on GCP. The use of ephemeral storage along with any cloud storage still requires the OPDB_USE_EPHEMERAL_STORAGE entitlement.This is available starting in COD v1.34 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed).

COD_ON_GCP entitlement has been removed from COD because it is not needed anymore. From this version onwards, customers can create COD clusters on Google Cloud Platform (GCP) without it.This is available starting in COD v1.34 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out the Cloudera Operational Database for free with Test Drive (no cloud account or setup is needed)

You can now get a consolidated view of all the COD metrics using the Grafana dashboard. To enable the Grafana dashboard, simply use the --enable-grafana option in the create-database command. This enables the Grafana URL under the GRAFANA DASHBOARD option inside your COD database. When you click on the Grafana URL, it takes you to the Grafana dashboard.Here is a sample command:cdp opdb create-database --environment <environment_name> --database <database_name> --enable-grafana   Important links:COD documentationRelease notes This is available starting in COD v1.33 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

You can define the --scale-type for both the master and gateway nodes using CDP CLI. The --scale-type option now supports all three options [--scale-type (string) <MICRO, LIGHT, HEAVY>] for both the --master-node-type and --gateway-node-type.--scale-type LIGHT (--master-node-type LITE, --gateway-node-type LITE)--scale-type HEAVY (--master-node-type HEAVY, --gateway-node-type HEAVY)If you do not define the --scale-type, <--scale-type LIGHT> is considered for both the --master-node-type and --gateway-node-type. However, you can overwrite the --scale-type for a --gateway-node-type using the [--gateway-node-type <value>] option. Important link:CDP CLI beta This is available starting in COD v1.33 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

You can upgrade the Cloudera Runtime version of the COD database using the rolling restart mode. This functionality ensures continuous service availability during an upgrade operation. This is achieved through a new CLI parameter --rolling-upgrade | --no-rolling-upgrade in the upgrade-database command.Here is a sample command:cdp opdb upgrade-database --environment <environment-name> --database <database-name> --runtime <runtime-version> [--rolling-upgrade | --no-rolling-upgrade] Important linksCOD documentationRelease notesThis is available starting in COD v1.33 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

COD now supports faster tolling restarts. This is achieved by increasing the default value of Region Mover Threads under Cloudera Manager > HBase > Configuration to 30. This ensures a smoother rolling restart for HBase. Important links:DocumentationRelease notesThis is available starting in COD v1.33 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

 We are pleased to announce the release of Cloudera ODBC Connector 2.7.0 for Apache Hive. This release has the following enhancements and fixes: Enhancements and New Features[HAR-1272] Updated Hive server version support[HAR-1350] Updated macOS support[HAR-1344][HAR-1349] [HAR-1351] Updated third-party libraries Issues Fixed[00420434] [HAR-1337] When in ZooKeeper discovery mode and a network error occurs, the connector does not connect to the same HS2 instance on a reconnecting attempt. [HAR-1325][HAR-1330] When executing a query with CREATE TABLE AS SELECT, the connector refers to a schema as an invalid table and returns an error. Known Issues[HAR-1171] The Hive server does not currently support using scientific notation for FLOAT type.On Hive 3.0 and later, when the server does not set the metastore schema reader properly, the driver returns an error. Workflow Changes[HAR-1350] Removed support for macOS universal bitness[HAR-1346] Updated SAML SSO c

The CDP Public Cloud Release Summary for the month of July is now available. It summarizes all major features introduced in CDP Public Cloud Management Console, Data Hub, and data services. Seehttps://docs.cloudera.com/cdp-public-cloud/cloud/release-summaries/topics/announcement-202307.html 

 We are pleased to announce the release of Cloudera JDBC Connector 2.6.32 for Impala.  This release has the following enhancements and fixes: Enhancements and New Features[IMPJ-740] Updated Jackson library: jackson-annotations 2.15.2 (previously 2.13.4)jackson-core 2.15.2 (previously 2.13.4)jackson-databind 2.15.2 (previously 2.13.4.2) Issues Fixed[02642284][IMPJ-719] On CHAR and VARCHAR types, the CAST statement fails. Getting Started with the Cloudera DriverRead the Cloudera JDBC Connector 2.6.32 for Impala Release Notes and Installation GuideDownload the connector from the Cloudera Connectors page As always, we welcome your feedback. Please send your comments and suggestions to the user group through our community forums. You can also file bugs through our external Jira projects on issues.cloudera.org.

You can now define custom recipes while creating an operational database using the --recipes option in CDP CLI Beta. You can register pre-created recipes during database creation based on the instance groups in your database. The recipes are executed automatically for the specified nodes based on the recipes’ type.Use the following example command to define custom recipes for your operational database. cdp opdb create-database --environment-name <ENVIRONMENT_NAME> --database-name <DATABASE_NAME> --recipe names=<rec1,rec2>,instanceGroup=<MASTER> names=<rec2>,instanceGroup=WORKER names=<rec3,rec4>,instanceGroup=<GATEWAY> To know more about the recipes, see Recipes.To know more about the --recipes option, see CDP CLI Beta. This is available starting in COD v1.32 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (n

We are excited to announce the general availability of Cloudera Streaming Analytics (CSA) 1.10.0 on CDP Private Cloud Base. This release includes a massive set of improvements to SQL Stream Builder (SSB), including the addition of built-in widgets for data visualization, as well as a rebase to Flink 1.16. Some of the key features of this release are: Rebase to Apache Flink 1.16 - Apache Flink 1.16 is now supported in CSA 1.10.PyFlink Support - The Python API for Flink is now supported in CSA. Customers can now create Flink DataStream applications using Python, besides Java and Scala, to build scalable batch and streaming workloads like real-time data processing pipelines, large-scale exploratory data analysis, Machine Learning (ML) pipelines and ETL processesBuilt-in Widgets for Data Visualization - Built-in data visualization widgets have been added to the SQL Stream Builder (SSB) UI to provide a quick and simple way to visualize data from streaming jobs and materialized vie

The CDP Public Cloud Release Summary for the month of June is now available. It summarizes all major features introduced in CDP Public Cloud Management Console, Data Hub, and data services. Seehttps://docs.cloudera.com/cdp-public-cloud/cloud/release-summaries/topics/announcement-202306.html 

SummaryCloudera Data Engineering (CDE) 1.19 introduces interactive Spark sessions for development workflows to take advantage of autoscaling compute and orchestration capabilities that's hybrid  and multi-cloud ready. Since there is no one size fits all approach to development, CDE interactive sessions give data engineers flexible end-points to start developing Spark applications from anywhere -- in a web-based terminal, local CLI, favorite IDE, and even via JDBC from third-party tools. CDE exposes sessions as first-class entities via the APIs, as well as the UI and CLI, allowing users to navigate seamlessly across interfaces. For example, initiate the session through the UI, start interacting with it in the web-based shell, then drop into your local terminal for a spark-shell experience.Interactive Sessions Video  Complete Feature List: Interactive Sessions (Tech Preview) Both CLI and web based interactive shell sessions are now supported. 

The latest release (2.5.0-b210) of Cloudera DataFlow (CDF) on Cloudera on cloud introduces NiFi 1.21, flow metrics driven auto-scaling, advanced UIs for JoltTransformJSON and UpdateAttribute processors, bulk actions in the Flow Designer, UDR support on Azure and makes in-place upgrades generally available to all customers. Flow Deployments and Test Sessions now support the latest Apache NiFi 1.21 release.In-place upgrades are now generally available to all customers. For more information, see Service upgrade.Flow Deployments now support Flow Metrics Scaling in addition to CPU utilization based auto-scaling. For more information, see Auto-scaling flow deploymentsWhen selecting Private Cluster during enablement on Azure, CDF no longer provisions a public load balancer and now supports User Defined Routes (UDR).Users can now provide their own values for the Kubernetes Pod CIDR Range and Kubernetes Service CIDR Range during enablementCDF now inherits load balancer configuration includ

On the COD UI, when you create an operational database the scale option Medium Duty is renamed to Heavy Duty. For more information, see Creating a database using COD. This is available starting in COD v1.32 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

In CDP CLI, when you select the --scale-type option as HEAVY, COD allocates larger SSD storage (for example, gp2 on AWS, StandardSSD_LRS on Azure, or pd-ssd on GCP) for both master and leader node types. For more information, see CDP CLI Beta. This is available starting in COD v1.32 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

 We're thrilled to announce the release of Cloudera Private Cloud (PVC) Data Services 1.5.1, the next iteration of Cloudera’s on-premises, cloud-native open data lakehouse, bringing the power of cloud-native data apps to your data center.  This release is jammed with 80+ new features across the platform and data services - Cloudera Data Engineering, Cloudera Data Warehouse, and Cloudera AI - to deliver faster insights on a more robust, secure, and performant platform.  Each of these are covered in detail in the release notes.   Some key platform highlights of this release include: Object storage with Cloudera Object Store (powered by Apache Ozone) is now GA on all data services, enabling modern data use-cases faster performance, denser storage, and better scalability.Open data format with Apache Iceberg V2 is available as a tech preview on all three data services, for greater flexibility and interoperability with your private open data lakehouse.