What's New @ Cloudera

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.14.2 (CDH 5.14.2, Cloudera Manager 5.14.2, and Cloudera Navigator 2.12.3).This release fixes key bugs and includes the following: The following upstream issues are fixed in CDH 5.14.2:HBASE-14252 - RegionServers fail to start when setting hbase.ipc.server.callqueue.scan.ratio to 0HDFS-13164 - File not closed if streamer fail with DSQuotaExceededException.HDFS-13040 - Kerberized inotify client fails despite kinit properlyHIVE-16886 - Hive metastore (HMS) log notifications may have duplicated event IDs if multiple HMSs are running concurrentlyHIVE-11609 - Capability to add a filter to hbase scan via composite key doesn't workHIVE-18127 - Do not strip '--' comments from shell commands issued from CliDriverHUE-8053 - [useradmin] LDAP authentication with sync_groups_on_login=true fails with KeyError exceptionHUE-7646 - [sentry] Changing role privileges removes

 Dear CDH, Cloudera Manager, and Cloudera Navigator users,We are pleased to announce the release of Cloudera Enterprise 5.13.3 (CDH 5.13.3, Cloudera Manager 5.13.3, and Cloudera Navigator 2.12.3).This release fixes key bugs and includes the following: The following upstream issues are fixed in CDH 5.13.3:HDFS-13170 - Port webhdfs unmaskedpermission parameter to HTTPFSHDFS-13040 - Kerberized inotify client fails despite kinit properlyHIVE-18526 HMS log notifications may have duplicated event IDs if multiple HMS are running concurrentlyHIVE-10815 causes HMS to generate events with duplicate eventIDsHUE-8053 - [useradmin] LDAP authentication with sync_groups_on_login=true fails with KeyError exceptionHUE-8055 - [desktop] Support multiple LDAP servers in LDAP Test commandHUE-8032 - [core] XSS in Hue Admin InterfaceHUE-7932 - [oozie] Clicking on workflow with several decision nodes hangs HueHUE-8024 - [indexer] Use old collection API to avoid grant Update privilegePARQUET-1217 - I

We are pleased to announce the release of Impala JDBC v2.5.43.  The release has the following fixes:  Resolved Issues in Impala JDBC 2.5.43When a parameter is bound to a BigDecimal Value, the driver rounds off the value incorrectly.In some cases, the driver incorrectly returns the following error: “Unable to obtain fetch semaphore in time. Fetch thread might have died unexpectedly”.Driver fails to connect through Kerberos when using a ticket cache with an IBM JVMIn some cases, the driver fails to execute batched insert operations using prepared statements that are written in Impala syntax.In some cases, the driver fails to execute batched update operations.Driver removes comments from queries when the UseNativeQuery property is set to 1. Getting Started with the Cloudera DriverRead the Cloudera Impala JDBC 2.5.43 Driver for Impala Release Notes and Installation Guide. Download the connector from the Cloudera Connectors page.  As always, we welcome your feedback

We are pleased to announce the release of Cloudera Director 2.7.1. Cloudera Director 2.7.1 provides bug fixes and and product improvements, as outlined in the Cloudera Director 2.7.1 release notes. For more info on downloading and installing Director, see the links below.Download Cloudera DirectorCloudera Director Release NotesGetting Started with Cloudera DirectorInstalling Cloudera Director Server and Client As always, we are happy to hear your feedback. Please post your questions and comments on our community forum.

 Dear CDH, Cloudera Manager, and Cloudera Navigator users,We are pleased to announce the release of Cloudera Enterprise 5.13.2 (CDH 5.13.2, Cloudera Manager 5.13.2, and Cloudera Navigator 2.12.2).This release fixes key bugs and includes the following: The following upstream issues are fixed in CDH 5.13.2:FLUME-3046 - Kafka Sink and Source Configuration ImprovementsHADOOP-14814 - Fix incompatible API change on FsServerDefaults to HADOOP-14104HBASE-14252 - RegionServers fail to start when setting hbase.ipc.server.callqueue.scan.ratio to 0HDFS-12052 - Set SWEBHDFS delegation token kind when ssl is enabled in HttpFSHIVE-18127 - Do not strip '--' comments from shell commands issued from CliDriverHIVE-15519 - BitSet not computed properly for ColumnBuffer subsetHIVE-14303 - CommonJoinOperator.checkAndGenObject should return directly to avoid NPE if ExecReducer.close is called twiceHIVE-11609 - Capability to add a filter to hbase scan via composite key doesn't workHUE-7830 - [core] N

Dear CDH, Cloudera Manager, and Cloudera Navigator users, On January 26, 2018, we announced the release of Cloudera Enterprise 5.14. Shortly after, we discovered two serious bugs in Cloudera Manager 5.14.  One results in the Cloudera Manager upgrade workflow incorrectly requiring the deployment of optional management roles.  The other causes Hive and HDFS Replication jobs to run slowly. Today we released Cloudera Manager 5.14.1 to fix these issues.  Please upgrade to this release instead of Cloudera Manager 5.14.0.  For more details, see the Fixed Issues section of the Release Notes.Please use the following information to assist in your implementation:Download Cloudera EnterpriseView the documentationAs always, we welcome your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects at issues.cloudera.org.

Cloudera is pleased to announce the release of Cloudera Data Science Workbench 1.3.0. This release includes fixes for some critical bugs. Details are in the Release Notes. For more information on downloading, installing and using, please see the links below:Download Cloudera Data Science WorkbenchCloudera Data Science Workbench OverviewGetting Started with Cloudera Data Science WorkbenchCloudera Data Science Workbench Release NotesInstalling Cloudera Data Science WorkbenchAs always, we welcome your feedback. Please send your comments and suggestions on our community forums.

Cloudera is pleased to announce that Cloudera Enterprise 5.14 is now generally available (GA). Our C5.14 release is delivering on the promises we’ve made to our customers, as Cloudera continues to innovate and set the pace for big data and machine learning platforms. This version brings to life key SDX components and improves the security profile of our platform, amongst many other product enhancements. Here are some selected highlights of CDH 5.14. As usual, there are also a number of quality enhancements, bug fixes, and other improvements across the stack. Here is a partial list of what’s included (see the Release Notes for a full list) Core PlatformNew data catalog and self-service discovery features in Cloudera Navigator enables searching and grouping of business metadata definitions for admins and users. Director support for Active Directory provides a tighter integration for authentication of users as clusters are provisioned and managed.Key Manageme

We are pleased to announce the release of Cloudera Director 2.7.0. Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise. What's New in Cloudera Director 2.7.0LDAP and Active Directory support for Cloudera Director users.Improved Java 8 support for clusters.Instance level control of normalization.AWS plugin improvements.A number of issues have been fixed. See Issues Fixed in Cloudera Director 2.7.0 for details. For more info on downloading and installing Director, see the links below. Download Cloudera DirectorCloudera Director Release NotesGetting Started with Cloudera DirectorInstalling Cloudera Director ClientInstalling Cloudera Director Server As always, we are happy to hear your feedback. Please post your questions and comments on our community f

We are pleased to announce the release of Impala JDBC v2.5.42.  The release has the following fixes and enhancements: Resolved Issues in Impala JDBC 2.5.42AllowSelfSignedCerts does not disable SSL. This issue has been resolved. Now, when the AllowSelfSignedCerts property is set to 1, SSL verification is disabled. The driver does not verify the server certificate against the trust store, and does not verify if the server's host name matches the CN or Subject Alternative Names in the server certificate. System does not display message for authentication failure. The com.cloudera.commons.codec.binary.Base64 class does not resolve to the correct dependency.If a connection fails due to there being no information available from the server, a generic system message is shown: “Error setting/closing session: null”. This been resolved, the driver now shows the appropriate connection error message in this scenario. Getting Started with the Cloudera DriverRead the Cloudera Impala JD

We are pleased to announce the release of Hive JDBC v2.5.20. The release has the following fixes and enhancements: Enhancements and New Features in Hive JDBC 2.5.20The driver now supports Apache Hive versions 0.11 through 2.1.The driver now supports SubjectAlternativeNames host name matching for SSL certificates.Authentication using Hadoop Delegation Tokens is now supported. Resolved Issues in Hive JDBC 2.5.20When next() is called for a second time on ResultSet that does not have any more rows, an incorrect exception is thrown. This issue has been resolved. The driver now returns False.AllowSelfignedCerts does not disable SSL. The issue has been resolved. Now, when AllowSelfSignedCerts property is set to 1, SSL verification is disabled. The driver does not verify the server certificate against the trust store, and does not verify if the server’s host name matches the CN or Subject Alternative Names in the server certificate.  This behavior is consistent with Cloudera JDB

We are happy to announce Apache Spark 2.2 release 2. You can download the parcel and apply it directly to provisioned clusters without disrupting your currently running Spark workloads. This component is generally available and is supported on CDH 5.8 and higher. A Hive compatibility issue in Cloudera Distribution of Apache Spark 2.0 release 1 affects CDH 5.10.1 and higher, CDH 5.9.2 and higher, CDH 5.8.5 and higher, and CDH 5.7.6 and higher. If you are using one of these CDH versions, you must upgrade to the Spark 2.0 release 2 or higher parcel, to avoid Spark 2 job failures when using Hive functionality. What's New in Cloudera Distribution of Apache Spark 2.2 Release 2This is purely a maintenance release. See Spark 2 Fixed Issues for the list of fixed issues. Issues Fixed in Cloudera Distribution of Apache Spark 2.2 release 2For a full list of fixed issues, see the list here. Download Cloudera Distribution of Apache Spark 2.2 release 2.Read the documentation.Want

Summary: Due to a kernel security exploit on system CPUs (Spectre and Meltdown), OS updates are required for all systems. The OS updates have performance implications. Cloudera is in the process of testing the impact across various Cloudera software services and workloads. Initial test results, based on the limited testing (a subset of services and workloads) we have completed as of January 12, indicate that the performance impact on Cloudera software is minor.   Users affected: While we are not aware of customers who have been impacted by these CPU vulnerabilities, we understand and expect that all Cloudera customers will apply Spectre and Meltdown patches as they become available from their OS suppliers. As such, Cloudera is performing basic functional testing to ensure that CDH works with the patches as well as determining performance impact on typical workloads.   Impact: Cloudera’s initial testing has focused on a subset of commonly used CDH serv

We are pleased to announce the release of Hive ODBC v2.5.25. The release has the following fixes and enhancements:Enhancements and New Features in Hive ODBC 2.5.25Updated Kerberos Configurations: The Host FQDM and Service Name configuration options (the KrbHostFQDN and KrbServiceName connection properties) are now optional. Also, the driver now automatically canonicalizes the server SPN when using Active Directory KerberosResolved Issues in Hive ODBC 2.5.25AD Kerberos authentication may use the wrong user when enabling constrained delegation in Tableau server on Windows.SQLGetInfo(SQL_KEYWORDS) returns an empty string. The driver now returns a comma-separated list of keywords natively defined by the data source except for the ones that are also ODBC reserved words. Getting Started with the Cloudera DriverRead the Cloudera ODBC 2.5.25 Driver for Hive Release Notes and Installation Guide.Download the connector from the Cloudera Connectors page.As always, we welcome your feedback. Please

We are pleased to announce the release of Cloudera Data Science Workbench 1.2.2. This release includes fixes for some critical bugs. Details are in the Release Notes. For more information on downloading, installing and using, please see the links below:Download Cloudera Data Science WorkbenchCloudera Data Science Workbench OverviewGetting Started with Cloudera Data Science WorkbenchCloudera Data Science Workbench Release NotesInstalling Cloudera Data Science WorkbenchAs always, we welcome your feedback. Please send your comments and suggestions on our community forums.

We are pleased to announce the release of Impala ODBC v2.5.36.0030.  The release has the following fixes and enhancements:Resolved Issues in Impala ODBC 2.5.36.0030In certain scenarios, the driver is not closing operation handle for query or metadata retrieval operations.Getting Started with the Cloudera DriverRead the Cloudera ODBC 2.5.36.0030 Driver for Impala Release Notes and Installation Guide. Download the connector from the Cloudera Connectors page. As always, we welcome your feedback. Please send your comments and suggestions to the user group of through our community forums. You can also file bugs through our external Jira projects on issues.cloudera.org.

 Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.12.2 (CDH 5.12.2, Cloudera Manager 5.12.2, and Cloudera Navigator 2.11.2).This release fixes key bugs and includes the following:CDH fixes for the following issues:FLUME-3046 - Kafka Sink and Source Configuration ImprovementsHBASE-18924 - Backport HBASE-18568 (Correct metric of numRegions) to branch-1.2 and branch-1.3HDFS-11738 - Hedged pread takes more time when block moved from initial locationsHDFS-12052 - Set SWEBHDFS delegation token kind when ssl is enabled in HttpFSHDFS-12642 - Log block and datanode details in BlockRecoveryWorker.HDFS-6763 - Initialize file system-wide quota once on transitioning to activeHDFS-12424 - Datatable sorting on the Datanode Information page in the Namenode UI is brokenHDFS-12369 - Edit log corruption due to hard lease recovery of not-closed file which has snapshots.HDFS-12363 - Possible NPE in BlockManager$StorageInfoDe

Dear CDH, Cloudera Manager, and Cloudera Navigator users,We are pleased to announce the release of Cloudera Enterprise 5.13.1 (CDH 5.13.1, Cloudera Manager 5.13.1, and Cloudera Navigator 2.12.1).This release fixes key bugs and includes the following:CDH fixes for the following issues:HBASE-18924 - Backport HBASE-18568 (Correct metric of numRegions) to branch-1.2 and branch-1.3HDFS-12424 - Datatable sorting on the Datanode Information page in the Namenode UI is brokenHDFS-12642 - Log block and datanode details in BlockRecoveryWorker.HIVE-17805 - SchemaTool validate locations should not return exit 1HIVE-16949 - Leak of threads from Get-Input-Paths and Get-Input-Summary thread poolHIVE-17218 - Canonical-ize hostnames for Hive metastore, and HS2 servers.HUE-6763 - [search] Add global default if in cloud modeHUE-7182 - [editor] Accented characters not inserted correctlyHUE-7521 - [security] Prevent JS error on printAcl not definedHUE-7489 - [editor] CloseOperation when GetOperationStatus i

We are pleased to announce the release of Cloudera Director 2.6.1. Cloudera Director 2.6.1 provides bug fixes and and product improvements, as outlined in the Cloudera Director 2.6.1 release notes. For more info on downloading and installing Director, see the links below.Download Cloudera DirectorCloudera Director Release NotesGetting Started with Cloudera DirectorInstalling Cloudera Director ClientInstalling Cloudera Director ServerAs always, we are happy to hear your feedback. Please post your questions and comments on our community forum.

We are pleased to announce the release of Impala ODBC v2.5.41.  The release has the following fixes and enhancements: Impala ODBC 2.5.41Enhancements and New Features in Impala ODBC 2.5.41The driver now supports CDH versions 5.0 through 5.12 Resolved Issues in Impala ODBC 2.5.41If a query fails due to “Memory Limit Exceeded”, the query is not closedCHAR field length reports incorrect values.Driver log file name does not include vendor. Getting Started with the Cloudera DriverRead the Cloudera ODBC 2.5.41 Driver for Impala Release Notes and Installation Guide. Download the connector from the Cloudera Connectors page.  As always, we welcome your feedback. Please send your comments and suggestions to the user group of through our community forums. You can also file bugs through our external Jira projects on issues.cloudera.org.

We are pleased to announce the release of the Cloudera Distribution of Apache Kafka 3.0.0 for CDH 5.  Apache Kafka is a highly scalable, distributed, publish-subscribe messaging system. Cloudera Distribution of Apache Kafka 3.0.0 is a major release based on Apache Kafka 0.11.0.0. Notable Issues Fixed in Cloudera Distribution of Apache Kafka 3.0.0:[KAFKA-5135] - Controller Health Metrics (KIP-143)[KAFKA-4422] - Drop support for Scala 2.10 (KIP-119)[KAFKA-3995] - Split the ProducerBatch and resend when received RecordTooLargeException[KAFKA-4815] - Idempotent/transactional Producer (KIP-98)[KAFKA-3878] - Exponential backoff for broker reconnect attempts (KIP-144)[KAFKA-4790] - Kafka cannot recover after a disk fullAll backported fixes can be viewed in the git release notes here or on our website under the Issues fixed section. We look forward to you trying Kafka 3.0.0. For more information, please use the links below:Install or upgrade KafkaReview the documentationRev

We are pleased to announce the release of Cloudera Director 2.6.0. Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise.  What's New in Cloudera Director 2.6.0 Expanded support for TLS:Automatic TLS for Cloudera Manager and CDH: Cloudera Director now provides the ability to automatically configure Cloudera Manager 5.13 or later and CDH 5.13 or later with TLS using a bespoke certificate authority based on the Cloudera Manager instance. See Automatic TLS (Auto-TLS).Manual TLS Support for Cloudera Manager: Cloudera Director is now able to communicate with Cloudera Manager 5.13 or later after it has been configured manually to communicate over TLS. See Manual TLS.TLS Support for Cloudera Director clients: The Java and Python client libraries and the Cloudera Direc

Cloudera is pleased to announce that Cloudera Enterprise 5.13 is now generally available (GA).  As usual, there are also a number of quality enhancements, bug fixes, and other improvements across the stack. Here is a partial list of what’s included (see the Release Notes for a full list): Core PlatformSDX Cloud Reference Architecture: Cloudera Enterprise now makes it possible for data management teams to provide a Shared Data Experience to disparate workloads running in the cloud. Key SDX capabilities include:Shared Data Catalog provides a unified data catalog experience for multi-cluster environments running in the cloud (with or without Cloudera Director) by sharing a common Apache Hive Metastore.Shared Security provides a unified authorization experience for multi-cluster environments running in the cloud (with or without Cloudera Director) by sharing a single instance of Apache Sentry.Shared Governance provides a unified governance and lineage experience for multi-cluster

We are pleased to announce the release of the Cloudera Distribution of Apache Kafka 2.1.2 for CDH 5.  Apache Kafka is a highly scalable, distributed, publish-subscribe messaging system. Apache Kafka 2.1.2 is maintenance release for Kafka 2.1.0. Notable Issues Fixed in Cloudera Distribution of Apache Kafka 2.1.2: KAFKA-3863: Add system test for connector failure/restart.KAFKA-3994: Deadlock between consumer heartbeat expiration and offset commit.All backported fixes can be viewed in the git release notes here or on our website under the Issues fixed section. We look forward to you trying Kafka 2.1.2. For more information, please use the links below:Install or upgrade KafkaReview the documentationReview the Release Notes As always, we welcome your feedback. Please send your comments and suggestions through our community forums.

We are happy to announce Apache Spark 2.1 release 2. You can download the parcel and apply it directly to provisioned clusters without disrupting your currently running Spark workloads. This component is generally available and is supported on CDH 5.7 and all subsequent CDH 5.x releases. Issues Fixed in Cloudera Distribution of Apache Spark 2.1 release 2[SPARK-22083][CORE] Release locks in MemoryStore.evictBlocksToFreeSpace[SPARK-20603][SS][TEST] Set default number of topic partitions to 1 to reduce the load[SPARK-21928][CORE] Set classloader on SerializerManager's private kryo[SPARK-20555][SQL] Fix mapping of Oracle DECIMAL types to Spark types in read path[SPARK-20848][SQL] Shutdown the pool after reading parquet filesFor a full list of fixed issues, see the list here.   Download Cloudera Distribution of Apache Spark 2.1 release 2.Read the documentation.Want to become a pro Spark user?  Sign up for Apache Spark Training.