Announcements

You can now run Data Hub clusters in your Google Cloud Platform account.  To make it easier to get started with GCP on Google Cloud, a Quick Start is available that will walk you through the prerequisite set up required in your cloud provider account, as well as creating a CDP credential and registering an environment in CDP. This will greatly reduce the amount of time it takes to set up a Proof of Concept for CDP.  CDP requires a number of pre-requisites to be set up in your GCP Project before an environment can be created.  The Quick Start includes GCP Deployment Manager templates to automate creation of pre-requisite resources including VPC network, a subnet, firewall rules, service accounts and storage buckets.   You can get started by reading the Quick Start on Google Cloud documentation.  Alternatively, if you'd prefer to set things up yourself, you can follow the documentation on working with a Google Cloud environment.&nbsp

New, lightweight and customizable Cloudera ML Runtimes are now available in CDP Machine Learning and Cloudera Data Science Workbench 1.9. ML Runtimes have been rebuilt from the ground up to enable greater flexibility in frameworks, processing, and IDEs — powering customizable lightweight deployments without over encumbering the runtime profile. The new profiles are designed to meet the diverse needs of Data Scientists by enabling a variety of ML Runtimes natively and eliminating sizing and versatility issues with previous Cloudera Engine profiles. ML Runtimes with version 2020-02 offer support for Python 3.6, 3.7, and 3.8 with both the Workbench editor and with JupyterLab (GA). All of these runtimes are offered as NVIDIA GPU edition with out-of-the-box GPU acceleration.New R 3.6 and 4.0 Runtimes are also available with the Workbench editor.  To learn more about our vision and roadmap read the Announcement blog post. As always, we welcome your feedback. Please send y

 We are pleased to announce the release of Cloudera JDBC 2.6.23 driver for Apache Impala.This release has the following fixes and enhancements: Enhancements and New Features[IMPJ-604] SSO authentication - You can now authenticate using single sign-on (SSO) via SAML 2.0 on supported servers. Workflow Changes[IMPJ-536] Removed support for JDBC 4.0 (Java 6) Getting Started with the Cloudera DriverRead the Cloudera Impala JDBC 2.6.20 Driver for  Apache Impala Release Notes and Installation Guide. Download the connector from the Cloudera Connectors page.  As always, we welcome your feedback. Please send your comments and suggestions to the user group through our community forums. You can also file bugs through our external Jira projects on issues.cloudera.org. 

 We are pleased to announce the release of Cloudera ODBC 2.6.13 driver for Apache Impala.  This release has the following enhancements and fixes: Enhancements and New Features[IMP-805][00263785] New authentication method[IMP-782] Updated authentication mechanism support Resolved Issues[IMP-846][00280482] When HTTP is enabled and two API calls with a time lag between them are made, the server resets the connection and the driver returns an error. Known Issues[IMP-843][00294390] The driver does not support query translation if the query contains the "LIMIT x OFFSET y" syntax.  Getting Started with the Cloudera DriverRead the Cloudera Impala ODBC 2.6.12 Driver for  Apache Impala Release Notes and Installation Guide. Download the connector from the Cloudera Connectors page.  As always, we welcome your feedback. Please send your comments and suggestions to the user group through our community forums. You ca

We are excited to announce that CDP Public Cloud Data Hub is now available on Google Cloud. This launch builds on our commitment to delivering an enterprise data cloud that enables our customers to do more with their data in a multi-cloud, hybrid deployment model. By providing the option to deploy CDP on Google Cloud, our customers will now have the flexibility and scale they need on the cloud platform of their choice.With CDP Public Cloud on Google Cloud, Cloudera customers can now use SDX and Data Hub to create secure data lakes in their Google Cloud account and provision Data Hub clusters in a matter of minutes.Today, CDP Public Cloud on Google Cloud delivers Data Hub with Data Engineering (Spark, Hive), Data Flow (NiFi) and Streams Management (Kafka) templates. Additional templates will be available in the coming months. In addition to the built-in cluster templates, customers can create custom cluster templates that combine any of the supported services. The combination of th

 CDS 3.1 powered by Apache Spark 3.1.1 is now generally available for CDP Private Cloud Base 7.1.6.This is a minor release of CDS 3. The main improvements include: The parcel contains spark3 compliant spark-hbase connectorAll performance enhancement of Apache Spark 3.1.1 such as new optimizer rules and improved subexpression eliminationsUnify create table SQL syntaxShuffled hash join improvement It can be installed as an add-on parcel on top of CDP Private Cloud Base 7.1.6. The bits can be found here and the latest documentation can be found here.  For Public Cloud the same bits will be available in a pre-warmed image for Cloudera Runtime 7.2.9. That can be installed through Data Engineering Spark3 Data Hub cluster templates. --------------Want to become a pro Spark user? Sign up for Apache Spark Training. 

We're pleased to announce the release of Cloudera Private Cloud (PvC) Base 7.1.6, which provides additional platform support and introduces key new features.  This is a cumulative maintenance release that carries forward the improvements from CDP PvC Base 7.1.5 and prior releases.   A key focus of this release has been to make migrations to CDP PvC Base easier, in addition to new features, and general platform support.  Here’s an overview of the new features and improvements we’ve added in this release.   Upgrade enhancementsTo help customers with their move to CDP PvC Base, we’ve made the following features available in this release: To support upgrading from HDP 3, we’ve released in-place upgrades for directly migrating your clusters to CDP PVC Base, and released a new version of the AM2CM tool (1.2.0) to support migrating from Ambari to Cloudera Manager.Documented rollback procedures for CDH 5 to CDP 7.1.6 and HDP 2 to CDP 7.1.6 are now availa

We are pleased to announce the release of Cloudera ODBC 2.6.12 driver for Apache Impala.  This release has the following enhancements and fixes: Enhancements and New Features[IMP-782][00189899] Updated support for the HTTP transport mode[IMP-801] Updated logging configurations[IMP-802] Updated OpenSSL support[IMP-780][IMP-819][IMP-821][IMP-825][IMP-827] [IMP-835] Updated platform support (for a list of supported platform versions, see the Installation and Configuration Guide)[IMP-807] Updated CDP support. Resolved Issues[IMP-806][00265194] When the server returns 0 rows, the driver stops fetching results.  Known IssuesThe following are known issues that you may encounter due to limitations in the data source, the driver, or an application. • BINARY is not a supported data type in Impala. • DATE is supported only when you connect to Impala version 3.3 or later. • As a result of the limitations above, some values of TIME data types are also not sup

We are pleased to announce the release of Cloudera JDBC 2.6.20 driver for Apache Impala.This release has the following fixes and enhancements: Enhancements and New Features[IMPJ-601] Updated third-party librariesThe JDBC 4.1 driver has been updated to use the following libraries:log4j 2.2.1slf4j 1.7.30[IMPJ-607] Updated CDP supportThe driver now supports CDP 7.1For a list of supported CDP versions, see the Installation and Configuration Guide Resolved Issues[00265194][IMPJ-606] When the server returns 0 rows, the driver stops fetching results[00276449][IMPJ-610] When executing consecutive statements, the driver does not update the parameter values[00276906][IMPJ-611] The driver incorrectly classifies queries containing comments and whitespaces  Known Issues The following are known issues that you may encounter due to limitations in the data source the driver or an application.The JDBC 4.0 version of the driver fails to connect to servers that require encryption usin

As part of the changes announced in October 2020, beginning in February 2021, no versions of Cloudera Manager, CDH, Apache Ambari, HDP and HDF releases will be available via Cloudera's public repositories.  Instead, all of the then-currently supported releases of Cloudera Manager, CDH, Ambari, HDP and HDF will be accessible only to active customers with valid subscription entitlements to these Cloudera products. This follows similar changes made for versions released since November 2019, which were never posted in Cloudera’s public repositories.     Additionally, new patch releases of Cloudera Manager and Ambari have been made available and are required to access Cloudera’s private repositories, which now contain these new and existing releases. Versions other than those outlined below will no longer be able to install CDH, HDP or HDF from Cloudera’s public repositories.  Customers should strongly consider upgrading to one of the new Cloudera Manager or Ambari

Text classification is a ubiquitous capability with a wealth of use cases including sentiment analysis, topic assignment, document identification, article recommendation, and more. But collecting enough annotated examples to train traditional classifiers can be quite costly. Instead, we take a look at a classic technique that can be used to perform text classification with few or even zero training examples! We're talking about text embeddings, of course. New advances have significantly increased the quality of document embeddings and in our newest writing on Few Shot Text Classification  we coverhow to use them for topic classification,best practices for using them,and potential limitations.Follow the links in the report to find code snippets so you can try it for yourself, and build your own demo so you can see the method in action!

 We are pleased to announce the release of Cloudera JDBC 2.6.13 driver for Apache Hive.The release has the following fixes and enhancements: Enhancements and New Features[HIVEJ-566] The driver now supports Apache Hive versions 1.0.0 through 3.1, and CDH versions 5.10 through 5.16 and 6.0 through 6.3. [HIVEJ-604] The driver now supports Java 11.  Resolved Issues[00184182][HIVEJ-602] The driver logs incorrect socket timeout values. [00255458][HIVEJ-637] When the connection is not valid, the driver does not notify the ConnectionEventListener object as expected. Known Issues The following are known issues that you may encounter due to limitations in the data source, the driver, or an application. [HIVEJ-577] The JDBC 4.0 version of the driver fails to connect to servers that require encryption using TLS 1.1 or later. When you attempt to connect to the server, the connection fails and the driver returns an SSL handshake exception. This issue occu

 Cloudera Fast Forward are happy to share recent updates of two of our older reports, now freely available to all!Semantic Image Search We recently revisited the topic of semantic search on image data. We’ve previously studied applications of deep learning to images in our early report Deep Learning for Image Analysis, and our more recent update: Deep Learning for Image Analysis: 2019 edition.In our most recent research cycle, we explored two critical requirements of semantic search at scale. First, we wrote a review of strategies for creating semantic representations of images (including supervised, self supervised, and unsupervised methods). Second, we provide an implementation of semantic search using fast approximate nearest neighbor search (with FAISS).We have released an updated version of ConvNet Playground App, and a set of scripts and tutorials for implementing semantic image search on the Cloudera Machine Learning platform.Federated Learning  Two years ago

New, lightweight and customizable Cloudera ML Runtimes are now generally available in CDP Machine Learning and Cloudera Data Science Workbench 1.9. ML Runtimes have been rebuilt from the ground up to enable greater flexibility in frameworks, processing, and IDEs — powering customizable lightweight deployments without over encumbering the runtime profile. The new profiles are designed to meet the diverse needs of Data Scientists by enabling a variety of ML Runtimes natively and eliminating sizing and versatility issues with previous Cloudera Engine profiles. This initial release ships with Python 3.6, 3.7, and 3.8 Runtimes as well as both the Workbench editor and with JupyterLab (Tech Preview). New Runtime options will soon follow! To learn more about our vision and roadmap read the Announcement blog post. Also in these releases: CDP Machine Learning:Support scaling down to zero CPUs or GPUs on AzureRefreshed Data Science Project dashboard experience CDSW 1.9:Av

We're pleased to announce the release of Cloudera Private Cloud (PvC) Base 7.1.5, which provides additional platform support and introduces key new features.  This is a cumulative maintenance release that carries forward the improvements from CDP PvC Base 7.1.4 and prior releases.   In this release, we’ve added the following new features and improvements: All core components of the platform are now FIPS Compliant.Improved Impala query performance using Bloom filters with a column predicate pushdown, allowing for optimized execution of filters. Clusters can now support ‘IGNORE’ operations in Apache Kudu.YARN Auto Queue Creation is now supported when Hive impersonation is turned off.Ranger support for enforcing Hive policies with HDFS access to Hive tables. Replication of Atlas metadata and Ranger policy for Hive databases (as a Tech Preview).Atlas audit now supports extended operations, such as type definitions, import, export, server start, and more.

Announcing Cloudera Enterprise 6.3.4 We are pleased to announce the release of Cloudera Enterprise 6.3.4, the world’s leading data platform for machine learning and analytics, optimized for the cloud. This maintenance release includes important bug fixes and CVE fixes since CDH-6.3.3.  Full release documentation is available here including the Cloudera Manager Release Notes and Cloudera Runtime Release Notes. New Features & CapabilitiesInstallation or Upgrade of Cloudera Manager and CDH requires authentication to access downloadsCloudera Express has been discontinuedIssues fixedUnsupported featuresIncompatible changesTimezone Names Unsupported in Impala in CDH 6.3.4Known issues and limitationsWhat's New in Cloudera Manager 6.3.4Fixed issues in Cloudera Manager 6.3.4Known Issues in Cloudera Manager 6.3.4 Important Note About Installation Beginning with Cloudera Manager 6.3.3 and CDH 6.3.3, downloading the software will require a valid Cloudera Ente

Cloudera fast forward are pleased to share our two latest applied machine learning research reports, on Meta-Learning and Structural Time Series. Read on here for more on each report,  check-out our blog post to read about our evolving research process, or head on over to our Research Roundup to view our latest research webinar on demand!Meta-Learning   In contrast to how humans learn, deep learning algorithms need vast amounts of data and compute and may yet struggle to generalize. Humans are successful in adapting quickly because they leverage their knowledge acquired from prior experience when faced with new problems. In this webinar we will explain how meta-learning can leverage previous knowledge acquired from data to solve novel tasks quickly and more efficiently during test time.Our report, Meta-Learning is freely available online, and accompanied by code that applies the technique to an image dataset. Read the report t

Model Monitoring & Deployment Security is now GA in CDSW 1.8. Available in this release is native MLOps functionalities for model monitoring, enabling tracking of individual model predictions down to the feature level for calculating model drift and ground truthing to production environments. Data scientists can analyze metrics using their preferred libraries and IDEs in any language, ensuring models are performing optimally and compliantly at scale. Additionally, this release includes general availability of Resource Quotas and quota management. These features enable administrators to limit users’ aggregate CPU, memory, and GPU consumption to protect against over-usage resulting in critical compute resource shortages. Default quotas can be configured for a Workspace, and overridden on a per-user basis with Custom quotas. Also in this release:Ability to use custom command-line arguments for jobs.Improved security for model deployments allowing user-level access controls t

We're pleased to announce the release of Cloudera Private Cloud (PvC) Base 7.1.4, which provides additional platform support and introduces key new features.  This is a cumulative maintenance release that carries forward the improvements from CDP PvC Base 7.1.3 and prior releases.   In this release, we’ve added the following new features and improvements: To help CDH and HDP customers upgrade to PvC Base, we’ve added platform support for Oracle 12, Oracle 19, SLES 12 SP5 and Postgres 11.New features to support your data warehouse with additional keywords, and replication between CDP PvC Base clusters.New features in Yarn Queue Manager to provide greater control with resource management.New security and governance features in SDX that provide multi-tenancy and translation of access control policies. See the 7.1.4 Release Summary for more details of these features.  CDH and HDP customers are encouraged to upgrade to CDP PvC Base for improved enterp