The ML Runtimes 2022.04-1 Release includes a technical preview version of the new workbench architecture, the PBJ (Powered by Jupyter) Workbench. In the previous Workbench editor, the code and output shown in the console (the right-hand pane in the image below) were passed to and from Python, R, or Scala via a Cloudera-specific, custom messaging protocol. In the PBJ Workbench, on the other hand, the code and output are now passed to and from the target language runtime via the Jupyter messaging protocol. They are handled inside the runtime container by a Jupyter kernel and rendered in your browser by JupyterLab’s client code.

This may seem like a subtle change, but it will provide CML users with some major benefits. First, the behavior of user code and third-party libraries on CML will be more consistent with its behavior in Jupyter-based environments. That means that a wider variety of rich visualization libraries will work out of the box, and in cases where rich visualization libraries do not work, error messages in the CML console and the browser console will be easier to google. Likewise, dependency conflicts between kernel code and user code will be rarer, and when they do occur they will be easier for customers to diagnose and fix. To give you a taste of what this higher degree of consistency is like, note that Python 3’s input() function now works. Go ahead and try it out!

Second, customers will no longer need to build runtime images starting from Cloudera base images and will no longer need to restrict themselves to languages and versions that Cloudera has packaged. Any combination of base image, target language, and language version can be used with the PBJ Workbench as long as a Jupyter kernel is available for that combination.

You can try it out by running a PBJ Workbench Python session using a CML April or newer Workspace. The look and feel of the workbench will be more or less unchanged. Under the hood, however, the way that code and outputs are rendered and passed between the web app and the Python interpreter have been re-engineered to better align with the Jupyter ecosystem.

The Technical Preview documentation is available here.

With the launch of CDP Public Cloud 7.2.15, Cloudera Streaming Analytics for Data Hub deployments has gotten some powerful new features with support for Dead Letter Queues.

With the launch of CDP Public Cloud 7.2.15, Cloudera Streams Messaging for Data Hub deployments has gotten some powerful new features! Streams Messaging now supports Multi-Availability Zone Deployments enabling High Availability, OAuth2 support for Clients connecting to Kafka Brokers and Schema Registry, Streams Messaging Manager Connect UI Changes, Kafka Connect security features, Debezium CDC Connectors, ability to import Kafka data to Atlas and much more.

With the launch of Cloudera Streaming Analytics 1.7.0 Parcel, Cloudera Streaming Analytics for Private Cloud deployments has gotten some powerful new features! A new User Interface, HA Capabilities for SQL Stream Builder, MSSQL CDC Support, and Materialized Views now support retention based on row counts.

The latest release of Cloudera DataFlow for the Public Cloud (CDF-PC 2.0.0-b302) adds support for NiFi flows that listen for incoming data (e.g. through ListenHTTP, ListenTCP, ListenSyslog processors) and allows users to deploy them in a cloud-native, Kubernetes based runtime with improved monitoring and auto-scaling capabilities. 

This release also introduces the latest Apache NiFi 1.16 version as well as additional new features, new ReadyFlows and stability improvements. 

Data warehouse users require constantly improving performance, regardless of data volumes or number of end users, with tools that are ever easier to use. Cloudera just released a capability that answers each of those calls, providing significant performance improvements that apply regardless of scale and that are almost trivial to set up. We are pleased to announce the general availability of Unified Analytics within Cloudera Data Warehouse (CDW) - our cloud native data warehouse service available in Cloudera Data Platform (CDP).

Cloudera Logging is a new Apache Log4j 1.2.x compatible logging library included with CDP Private Cloud Base 7.1.7 SP1 that is created, distributed, and maintained by Cloudera to address the recent vulnerabilities in Apache Log4j 1.2.x.

What does Cloudera's new Long Term Support (LTS) release for mean for customers?  Do you stand to benefit from it?   In this article we'll answer some of the common questions around  the LTS release for CDP Private Cloud Base. 

You can now download the Phoenix Client Jars with a single click directly from the Phoenix Thick client and Phoenix Thin client tabs in the UI.

With the launch of CDP Public Cloud 7.2.14, Cloudera Streams Messaging for Data Hub deployments has gotten some powerful new features! In this release, the Streams Messaging templates in Data Hub will come with Apache Kafka 2.8 and Cruise Control 2.5 providing new core features and fixes. KConnect has been added and gains additional capabilities with new connectors and Stateless Apache NiFi capabilities which can run NiFi Flows as connectors.  The Schema Registry will now support JSON schemas in addition to the Apache Avro schemas already supported and will gain the ability to perform native API based import and export to share schemas between environments. 

COD now supports the “Storefile Tracking” (SFT) as an optional feature in Runtime 7.2.14.0.

COD allows to disable the Kerberos authentication temporarily for HBase clients that run on Cloudera legacy products.

COD supports custom table coprocessors, which you can implement and extend from HBase coprocessors’ interfaces.

COD supports RAZ integration from the Runtime version 7.2.11.0. You can grant fine-grained access to directories.

For a smooth and better functionality, COD is now bundled with the HBase version 2.4.6. You need to upgrade the HBase client version for seamless connectivity.

The transaction support in COD enables you to perform complex distributed transactions and run atomic database operations, meaning your database operations must either be completed or terminated. A transaction ensures adhering to the ACID properties.

Starting with the February release (v1.14) of CDE , Apache Iceberg tables are now supported with Spark 3 virtual clusters on AWS. Users can query and work with tables at petabyte scale without impacting query planning, while benefiting from efficient metadata management, snapshotting, and time-travel.

The latest release of Cloudera DataFlow for the Public Cloud (CDF-PC 1.1.0-b127) allows Microsoft Azure users to run their Apache NiFi flows in a cloud-native, Kubernetes based runtime with improved monitoring and auto-scaling capabilities. 

This release also introduces the following new features for both, AWS and Azure customers:

• Flow Deployments now support the latest Apache NiFi 1.15 release.
• CDF now creates Kubernetes clusters with version 1.20 in AWS EKS and Azure AKS.
• CDF now supports username/password authentication for AWS environments with non-transparent proxy setups.
• When using the Default NiFi SSL Context Service in a flow deployment, the automatically generated truststore now contains the default cacerts from the local JDK. This ensures that the SSL Context Service can be used with 3rd party applications using certificates from common public certificate authorities.
• Users can now perform the Download Kubeconfig and Manage User Access actions when an enablement request fails. This allows in-depth troubleshooting of a failed enablement attempt.
  
• The following ReadyFlows have been added to the ReadyFlow Gallery:
  ADLS to ADLS Avro ReadyFlow
  Kafka to ADLS Avro ReadyFlow
  
  
  Check out the Announcement blog post and learn more about the new features in the CDF-PC documentation and take an interactive tour of CDF-PC
  

COD now supports 1.6TB NVMe (Non-volatile Memory Express) based cache that significantly improves the performance when you deploy COD with object storage.

COD is now available on GCP as Technical Preview, in addition to AWS and Azure.

COD now provides consistently high performance by deploying HDFS to store data.

CDP Operational Database (COD) now supports configuring a storage instance to cache HBase data stored in S3.

CDP Operational Database (COD) supports copying tables from one COD cluster to another.

Cloudera supports upgrading an installed runtime version to an advanced one with downtime.

CDP Operational Database (COD) supports disabling the autonomous functions.

CDP Operational Database (COD) now automatically sets configuration properties for the Cloudera replication plugin.

CDP Operational Database (COD) now supports custom user tags while using CDP CLI Beta.

CDP Operational Database (COD) now supports and deploys HDFS on Cloud Block Storage. COD now provides an updated ‘update-database’ command in CDP CLI Beta.

Cloudera now supports Phoenix access through the Apache Hue SQL assistant.

With CML's multi-version Spark support, CML users can now run different versions of Spark side by side, even within a single project.