Cloudera is excited to announce the general availability of Cloudera Enterprise 5.9!
Cloudera Enterprise 5.9 contains a long list of new features, quality enhancements, bug fixes, and other improvements across the stack. Here is a partial list of those improvements; see the Release Notes for a full list:
What's New In CDH 5.9.0 Apache Hadoop
- You can use temporary credentials to log in to Amazon S3 and obtain temporary credentials from Amazon's Security Token Service (STS).
Apache HBase
- A tool has been added to dump existing replication peers, configurations, and queues when using HBase replication. For more information, see Class DumpReplicationQueues.
- Metrics have been added that expose the amount of replayed work occurring in the HBase replication system. For more information, see Replication Metrics in the Apache HBase Reference Guide.
Apache Hive
Hue
- HUE-2915: Integrates Hue with Amazon S3. You can now access both S3 and HDFS in the File Browser, create tables from files in S3, and save query results in S3. See how to Enable S3 Cloud Storage.
- HUE-4039: Improves SQL Autocompleter. The new Autocompleter understands Hive and Impala SQL dialects and provides smart suggestions based on statement structure and cursor position. See how to manually Enable and Disable Autocompleter.
- HUE-3877: Adds support for Amazon RDS. You can now deploy Hue against an Amazon RDS database instance with MySQL, PostgreSQL, and Oracle engines.
- Rebase of Hue on upstream Hue 3.11.
Apache Impala (incubating)
- Performance improvements:
- [IMPALA-1683] The REFRESH statement can be applied to a single partition, rather than the entire table. See REFRESH Statement and Refreshing a Single Partition for details.
- Improvements to the Impala web user interface:
- [IMPALA-2767] You can now force a session to expire by clicking a link in the web UI, on the /sessions tab.
- [IMPALA-3715] The /memz tab includes more information about Impala memory usage.
- [IMPALA-3716] The Details page for a query now includes a Memory tab.
- [IMPALA-3499] Scalability improvements to the catalog server.
- [IMPALA-3677] You can send a SIGUSR1 signal to any Impala-related daemon to write a Breakpad minidump. See Breakpad Minidumps for Impala (CDH 5.8 or higher only) for details about the Breakpad minidump feature.
- [IMPALA-3687] The schema reconciliation rules for Avro tables have changed slightly for CHAR and VARCHAR columns. See Creating Avro Tables for details about column definitions in Avro tables.
- [IMPALA-3575] Some network operations now have additional timeout and retry settings.
Apache Sentry
- Sentry adds support for securing data on Amazon RDS. As a result, Sentry can secure URIs with an RDS schema.
- SENTRY-1233 - Logging improvements for SentryConfigToolSolr.
- SENTRY-1119 - Allow data engines to obtain the ActionFactory directly from the configuration, instead of having hardcoded component-specific classes.
- SENTRY-1229 - Added a basic configurable cache to SentryGenericProviderBackend.
Apache Spark
- You can now set up AWS credentials for Spark with the Hadoop credential provider, to avoid exposing the AWS secret key in configuration files.
Apache Sqoop
- The mainframe import module extension has been added to support data sets on tape.
Cloudera Search
- The Solr watchdog is now configured to use the fully qualified domain name (FQDN) of the host on which the Solr process is running (instead of 127.0.0.1). You can override this configuration by setting the SOLR_HOSTNAME environment variable to appropriate value.
- Cloudera Search adds support for index snapshots. For more information, see Backing Up and Restoring Cloudera Search.
What's New in Cloudera Manager 5.9.0
- You can create virtual images of Cloudera Manager and cluster hosts. See Creating Virtual Images of Cluster Hosts.
- Security
- External/Cloud account configuration in Cloudera Manager.
- Account configuration for access to Amazon Web Services is now available through the centralized UI menu External Accounts.
- Key Trustee Server rolling restart.
- Key Trustee Server now supports rolling restart.
- Backup and Disaster Recovery
- Resource Management
- You can create custom Cluster Utilization reports that you can export data from. See Creating a Custom Cluster Utilization Report.
- When Cloudera Manager manages multiple clusters, Historical Applications and User and Historical Queries by User show applications per user and per pool.
- Directory usage reports can be exported as a CSV file.
- Cloudera Manager API
- Added the update_user() method to the Python API client api_client.py.
- New API endpoints have been added that allow users to add, list and remove Watched Directories in HDFS service.
- Logging
- Kafka log4j log files now include the hostname in the format kafka-broker-${host}.log. Similarly, MirrorMaker logs now include the hostname in the format kafka-mirrormaker-${host}.log.
- Cloudera Manager displays the History and Rollback support for the Cloudera Manager Settings. This helps Cloudera Support provide better service when certain Cloudera Manager administrative settings are modified.
- Diagnostic Bundles
- You can specify information to be redacted in the diagnostic bundle in the UI using Administration > Settings > Redaction Parameters for Diagnostic Bundles.
- Upgrade
- Informs you when a simple restart is performed instead of rolling restart on a service because rolling restart is not available.
- Oozie
- The Actions menu in the Oozie service has two new commands, Dump Database and Load Database, which make it easier to migrate an Oozie database to another database supported by Oozie.
- Install Oozie ShareLib command assigns correct permissions to the uploaded libraries. This prevents breaking Oozie workflows with a custom umask setting.
- Configuration Changes
- Added the zkClientTimeout parameter for ZooKeeper.
- Added a new option for setting the file format used by an ApplicationMaster when generating the .jhist file.
- Adds graceful decommission on YARN NodeManager roles. The NodeManager is not assigned new containers, and it waits for any currently running applications to finish before being decommissioned, unless a timeout occurs. Configure the timeout using the Node Manager Graceful Decommission Timeout configuration property in the YARN Service.
- stdout and stderr log links are now shown in the UI when a failure occurs while deploying client configurations.
- Added the configuration parameter, Extra Space Ratio for Indexing, to Reports Manager. Use the parameter to increase indexing speed by allocating additional memory.
- The default amount of time that HBase Indexer roles attempt to connect to ZooKeeper has been increased from 30 to 60 seconds.
- Cloudera Manager can identify whether or not a customer is using the embedded PostgreSQL database. If Cloudera Manager is configured to use the embedded PostgreSQL database, a yellow banner appears in the UI, recommending that you upgrade to a supported external database.
- When Impala uses SSL, TLS Connection to Catalog Server is now supported. You can enable replication for any Impala UDFs/Metadata (in Hive Replication).
- When running wizards from the Cloudera Manager Admin Console that add a cluster, add a service, perform an upgrade, and other tasks, steps do not display when they are not reachable or do not apply to the current configuration.
- Improve Cloudera Manager provisioning performance on AWS.
- Add support for resetting Cloudera Manager GUID/UUID by checking the UUID file.
Over the next few weeks, we will publish blog posts that cover some of these features in detail. In the meantime:
Download Cloudera Enterprise 5.9
Explore documentation
As always, we value your feedback; please provide any comments and suggestions through our community forums. You can also file bugs through issues.cloudera.org.
