We are pleased to announce the general availability of Cloudera Enterprise 6.1.0, the modern platform for machine learning and analytics optimized for the cloud. This release delivers a number of new capabilities, improved usability, and better performance.
New capabilities include:
- Management
- Cloudera Manager can test the network latency of all network links within a cluster or between clusters to identify performance bottlenecks or misconfigured networks
- BDR now supports backup from non-secure (non-Kerbrerized) clusters to secure (Kerberized) clusters
- Impala health monitoring diagnostics have been significantly improved
- New guardrails for Impala query memory usage are available for Impala resource pools, improving multi-tenancy for Impala
- Storage
- Support for HDFS Erasure encoding for Hive, Navigator, BDR, MapReduce and Spark workloads can reduce storage requirements by up to 50% with a negligible performance impact
- Support for Azure Data Lake Storage Gen 2 enables better performance and lower cost for customers deploying CDH to Azure
- Search, query, access
- Impala now supports exact multiple COUNT(DISTINCT <expr>) within a single query allowing more complex data warehouse queries to be run
- Ingest
- Support for Spark Structured Streaming - which enables micro-batch processing at as little as 100ms increments with SQL-like APIs - including DataFrames and Datasets, while simplifying implementations via abstractions.
- Flume now supports continuous ingest of data into Kudu (from messaging sources such as Kafka, JMS or Avro) using the Flume Kudu sink
- Kafka now supports JBOD enabling customers to use cheaper disk and reduce the cost of storage
- Sqoop now supports loading data into S3 and permits creating tables & loading data directly in a Sentry-secured Hive database with a single step
- Security - Finer-grained permissions (also in C5.16.1)
- Sentry adds CREATE permission and user-level ownership of tables. This enables secure sharing of a single sandbox database among many users and eliminates the administrative overhead of creating separate databases, roles, and groups to preserve privacy for one person or a small group.
- Impala REFRESH METADATA permission, allows admins to regulate who can execute this impactful Impala command.
- Security - Key Management
- Support for AWS CloudHSM enables customers deploying HDFS clusters on AWS to protect encryption keys in isolated purpose-built hardware security modules.
- Platform Support
- Support for deploying with OpenJDK 8.
- MapReduce now supports the Zstandard compression codec
Usability enhancements include:
- Search, query, access
- Data discovery simplifications in Hue help users get to the right data faster
- Query queuing visualizations in Hue identify when clusters are busy, preventing user frustration and multiple resend of queries to busy clusters
- Ingest
- Flume agents now have push-button wire encryption using TLS with Cloudera Manager 6’s Auto-TLS feature
- Simplified access control, improved security defaults and better metrics in Kafka
- Governance - Navigator
- Autocomplete for Databases, Tables/Views, and Fields - simplify searches.
- Hive table details page shows more details on each column - allows Navigator users to see the details about all columns of a table on a single screen.
Performance enhancements include:
- Improved scanning performance, and compactions rate limits in Accumulo
- Governance
- Navigator handles larger volumes of data with more select HDFS event and metadata capture.
Please refer to the release notes for a complete list of features. We also encourage you to review the new Upgrade Guide that now includes the ability to create a customized document based on your unique upgrade path. In particular, Cloudera Manager 6.1.0 supports upgrading CDH 5.15.0 and CDH 5.16.1 clusters to CDH 6.1.0.
Cloudera Enterprise 6.1.0 includes updated versions of many of our platform components, including rebases to the following Apache project versions: Kafka 2.0, Spark 2.4, HBase 2.1.1, Accumulo 1.9.2, and Solr 7.4
Additional information is available in the documentation.
As always, we'd love your feedback and remain committed to your success! Please provide any comments and suggestions through our community forums.
