What's New @ Cloudera

If you would like to review CDP Public Cloud features released in March 2022, see CDP Public Cloud: March 2022 Release Summary.Feel free tc comment on this article to let us know how we can make these release summaries more useful. 

We're pleased to announce the release of Cloudera Private Cloud Base 7.1.7 Service Pack 1 (SP1) and Cloudera Manager version 7.6.1.  This is the first service pack we’ve released for CDP Private Cloud Base, and with it we announce that CDP Private Cloud Base 7.1.7 has been designated a Long Term Supported (LTS) release.   The LTS release provides a stable version of the platform to be supported for 4 years.  There will be periodic cumulative hotfixes released to address any vulnerabilities or bug fixes to ensure this version remains secure, however, there won’t be any major changes or new features that require significant testing.  This enables customers with complex cluster implementations or workloads supporting critical business operations that need extensive testing to upgrade with confidence to a version of CDP Private Cloud Base that will remain stable for longer.  Customers on LTS need only upgrade when the next LTS version is released. &n

Check out the blog detailing the latest features here. https://blog.cloudera.com/new-features-in-cloudera-streams-messaging-for-cdp-public-cloud-7-2-14/

 We are pleased to announce the release of Cloudera JDBC Driver 2.6.18 for Apache Hive. This release has the following enhancements and fixes: Enhancements and New FeaturesNone Issues Resolved[00372375] [HIVEJ-728] When close() is not called on the connection object, the connector prevents the JVM from exiting. Getting Started with the Cloudera DriverRead the Cloudera JDBC Driver 2.6.18 for Apache Hive Release Notes and Installation GuideDownload the connector from the Cloudera Connectors page As always, we welcome your feedback. Please send your comments and suggestions to the user group through our community forums. You can also file bugs through our external Jira projects on issues.cloudera.org.

Storefile Tracking (SFT) changes how HBase manages its files to avoid operations which are known to be suboptimal when using object stores. COD enables this feature for COD databases deployed on AWS which use S3 for HBase storage which will address performance issues known around flushes, compactions, and other HBase operations. This feature is in Preview and available on new COD services only. For more information, see HBase Storefile Tracking. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

If your client applications are running on Cloudera legacy products, they usually do not have Kerberos authentication enabled. When you try to connect to any COD instance, the connection fails because the COD instances have Kerberos enabled, by default. Now, you can disable Kerberos authentication in your COD instances so that HBase or Phoenix clients can connect seamlessly. For more information, see Disabling Kerberos authentication for HBase clients. This is available starting in COD v1.19 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

You can add table coprocessors so that HBase can run custom code on the server side against the stored data and filter local minimum or maximum value during ingestion without scanning the entire table. You can use built-in table coprocessors from the upstream HBase releases. This feature is in Preview and available on new COD services only. For more information, see Working with custom table coprocessors. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

The Ranger Authorization Service (RAZ) is a fine grained authorization service for cloud storage. As a regular individual user or as an HBase user, you can limit the authorization levels in the cloud storage to a directory level. For more information, see COD integration with RAZ . This is available starting in COD v1.19 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

COD is now bundled and shipped along with the HBase version 2.4.6 when the CDP Runtime version is 7.2.14. This is available starting in COD v1.19 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

COD supports Apache OMID transactional framework that allows Big Data applications to execute ACID transactions on top of Phoenix tables. This is available starting in COD v1.19 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

If you would like to review CDP Public Cloud features released in February 2022, see CDP Public Cloud: February 2022 Release Summary.Feel free tc comment on this article to let us know how we can make these release summaries more useful. 

 We are pleased to announce the release of Cloudera JDBC Driver 2.6.17 for Apache Hive. This release has the following enhancements and fixes: Enhancements and New Features[HIVEJ-704] Updated log4j third-party libraries Issues Resolved[HIVEJ-672] The HiveJDBCDataEngine does not check getFunctions and returns an empty result set.[HIVEJ-684] When a ZooKeeper connection URL does not contain a namespace for each host, the connector does not close the connection properly.  Getting Started with the Cloudera DriverRead the Cloudera JDBC Driver 2.6.17 for Apache Hive Release Notes and Installation GuideDownload the connector from the Cloudera Connectors page As always, we welcome your feedback. Please send your comments and suggestions to the user group through our community forums. You can also file bugs through our external Jira projects on issues.cloudera.org. 

 We are pleased to announce the release of Cloudera JDBC 2.6.26 driver for Impala.  This release has the following enhancements: Enhancements and New Features[IMPJ-673] Updated log4j third-party libraries Getting Started with the Cloudera DriverRead the Cloudera JDBC Connector 2.6.26 for Impala Release Notes and Installation GuideDownload the connector from the Cloudera Connectors page As always, we welcome your feedback. Please send your comments and suggestions to the user group through our community forums. You can also file bugs through our external Jira projects on issues.cloudera.org.

Apache Iceberg is a new open table format targeted for petabyte-scale analytic datasets.  It  has been designed and developed as an open community standard to ensure compatibility across languages and implementations.  The integration of Iceberg with CDP's multi-function analytics and multi-cloud platform, provides a unique solution that future-proofs the data architecture for new and existing Cloudera customers.  Users can define and manipulate datasets with SQL commands, build complex data pipelines using features like Time Travel operations, and deploy machine learning models built from Iceberg tables. By the end of the month, besides using Iceberg with ETL/ELT workloads, we will extend multi-analytic workloads to Iceberg tables in Cloudera Data Warehouse (CDW) with Hive and Impala for interactive BI and SQL analytics. This feature is in Preview and available on new CDE services only.  Learn more here

The latest release of Cloudera DataFlow for the Public Cloud (CDF-PC 1.1.0-b127) allows Microsoft Azure users to run their Apache NiFi flows in a cloud-native, Kubernetes based runtime with improved monitoring and auto-scaling capabilities.  This release also introduces the following new features for both, AWS and Azure customers:Flow Deployments now support the latest Apache NiFi 1.15 release.CDF now creates Kubernetes clusters with version 1.20 in AWS EKS and Azure AKS.CDF now supports username/password authentication for AWS environments with non-transparent proxy setups.When using the Default NiFi SSL Context Service in a flow deployment, the automatically generated truststore now contains the default cacerts from the local JDK. This ensures that the SSL Context Service can be used with 3rd party applications using certificates from common public certificate authorities.Users can now perform the Download Kubeconfig and Manage User Access actions when an enablement request

CDP Operational Database (COD) now supports the configuration of instance storage to cache HBase data stored in block storage. This is only available on AMD instance types. This is available starting in COD v1.18 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

COD is now available on GCP as Technical Preview, in addition to AWS and Azure. Now, you can utilize Cloudera’s Transactional Database capabilities on GCP for both HBase and Phoenix customers. You can lift and shift the databases to GCP or enable hybrid cloud use cases between on-prem and GCP.You can create a COD instance on GCP using the CDP CLI Beta. Cloudera is providing the support for COD on GCP starting in COD v1.17.0.Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed), or reach out to your account team for how to participate in the tech preview program.

COD now provides HDFS as a storage option for COD deployment instead of cloud storage. This is available starting in COD v1.14 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

COD now supports configuration of instance storage to cache HBase data stored in S3. This is available starting in COD v1.14 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

COD provides a CopyTable utility to copy tables from one COD cluster to another. This is available starting in COD v1.14 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

You can upgrade the Cloudera Runtime installed on the cluster running COD to an advanced one with downtime. This is available starting in COD v1.14 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

COD can disable the autonomous database functions, and because of that, the autonomous functions stop managing a database. A database cannot leverage an autonomous function when COD disables it; however, the functions are available and accessible as a datahub cluster. This is available starting in COD v1.14 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

When you create a COD database, COD automatically sets configuration properties for the Cloudera replication plugin so that on-prem clusters can replicate into COD databases. COD now automatically configures the OpDB replication plugin and reduces the time needed to move the application into production. This is available starting in COD v1.13 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

When you create a COD database, COD enables you to supply custom tags for launched hardwares while using CDP CLI Beta. This is available starting in COD v1.13 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).

COD now supports HDFS for fast and consistent performance.Cloud Object Storage has a high variance in latency, and sometimes the delay in accessing the data is high. The operational cost in managing such an operational database is expensive, as it reads millions of data per second. COD now deploys HDFS on Cloud Block Storage, which ensures that the performance characteristics are similar to on-prem. This feature simplifies the lift-and-shift efforts to the cloud.This capability includes an updated ‘update-database’ command in CDP CLI Beta to enable tuning of auto-scaling based on HDFS utilisation. This is available starting in COD v1.13 and you do not need to upgrade to a higher version of COD to benefit from this feature. Try out CDP Operational Database for free with Test Drive (no cloud account or setup is needed).