Announcements

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.10.2 (CDH 5.10.2, Cloudera Manager 5.10.2, and Cloudera Navigator 2.9.2). This release fixes key bugs and includes the following: CDH fixes for the following issues:HADOOP-12751 - While using kerberos Hadoop incorrectly assumes names with '@' to be non-simpleHBASE-16630 - Fragmentation in long running Bucket CacheHIVE-12768 - Thread safety: binary sortable serde decimal deserializationHIVE-14564 - Column Pruning generates out of order columns in SelectOperator which cause ArrayIndexOutOfBoundsExceptionIMPALA-4725 - Query option to control Parquet array resolutionIMPALA-4546 - Fix Moscow timezone conversion after 2014IMPALA-4822 - Implement dynamic log level changesIMPALA-5487 - Race in runtime-profile.cc::toThrift() can lead to corrupt profiles being generated while query is runningFor a full list of upstream JIRAs fixed in CDH 5.10.2, see the issues

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.11.1 (CDH 5.11.1, Cloudera Manager 5.11.1, and Cloudera Navigator 2.10.1). This release fixes key bugs and includes the following: CDH fixes for the following issues:HADOOP-14028 - S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failuresHIVE-12768 - Thread safety: binary sortable serde decimal deserializationIMPALA-5251 - DecimalAvgFinalize() gets the wrong arg typeIMPALA-5186 - Handle failed CreateAndOpenScanner() in MT scan.IMPALA-3641 - DROP / CREATE sequence on same table failed with "table already exists".IMPALA-4902 - Concurrent DDL may fail with a ConcurrentModificationException.IMPALA-5253 - Use appropriate transport for StatestoreSubscriberIMPALA-5145 - CTAS failing when creating from a view with error "Unsupported type 'null_type'"For a full list of upstream JIRAs fixed in CDH 5.11.1, see the

Hello CDH Users, We are pleased to announce the release of Impala JDBC v2.5.38. The release has the following fixes: Resolved IssuesDriver incorrectly passes SSLTrustStore configuration to the server as server-side property. Translation fails due to the driver treating coercion between STRING and BIGINT columns as an invalid operation.When certain fetch sizes are set for the result set, the driver incorrectly returns an error while executing a query.  Getting Started with the Cloudera DriverRead the Cloudera JDBC 2.5.38 Driver for Impala Release Notes and Installation Guide.Download the connector from the Cloudera Connectors page. As always, we welcome your feedback. Please send your comments and suggestions to the user group of through our community forums. You can also file bugs through our external Jira projects issues.cloudera.org.

Cloudera is happy to announce the availability of parcels and packages for Apache Kudu 1.3.0- CDH 5.11.1. Apache Kudu 1.3.0- CDH 5.11.1 is a bug fix release, with no new features or backwards incompatible changes. To upgrade Kudu to 1.3.0- CDH 5.11.1, see Upgrade Parcels or Upgrade Packages. Issues Fixed in Kudu 1.3.0- CDH 5.11.1 KUDU-1999 - Fixed an issue where the Kudu Spark connector would fail to kinit with the principal and keytab provided to a job.KUDU-1993 - Fixed a validation issue with grouped gflags.KUDU-1981 - Fixed an issue where Kudu server components would fail to start on machines with fully-qualified domain names longer than 64 characters when security was enabled. This was due to hard-coded restrictions in the OpenSSL library.KUDU-1607 - Fixed a case in which a tablet replica on a tablet server could retain blocks of data which prevented it from being fully deleted.KUDU-1933 - Fixed an issue in which a tablet server would crash and fail to restart after

We are pleased to announce the release of Cloudera Director 2.4.1. Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise.  Cloudera Director 2.4.1 provides bug fixes and and product improvements, as outlined in the Cloudera Director 2.4.1 release notes. Download Cloudera DirectorCloudera Director Release NotesGetting Started with Cloudera DirectorInstalling Cloudera Director ClientInstalling Cloudera Director Server As always, we are happy to hear your feedback. Please post your questions and comments on our community forum.

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.8.5 (CDH 5.8.5, Cloudera Manager 5.8.5, and Cloudera Navigator 2.7.5.). This release fixes key bugs and includes the following: CDH fixes for the following issues:FLUME-2798 - Malformed Syslog messages can lead to OutOfMemoryExceptionFLUME-2812 - Fix semaphore leak causing java.lang.Error: Maximum permit count exceeded in MemoryChannelHADOOP-13579 - Fix source-level compatibility after HADOOP-11252HDFS-11229 - HDFS-11056 failed to close meta fileHIVE-11487 - Add getNumPartitionsByFilter api in metastore apiHIVE-12465 - Hive might produce wrong results when (outer) joins are mergedHIVE-12768 - Thread safety: binary sortable serde decimal deserializationHUE-3065 - [oozie] Sub-workflow submitted from coordinator gets parent workflow graphHUE-4147 - [useradmin] Ignore (objectclass=*) filter when searching for LDAP usersIMPALA-3018 - Don't return NULL on z

Dear Cloudera User,   We are excited to announce the General Availability (GA) release of Cloudera Altus.  Cloudera Altus is a platform-as-a-service offering that makes it dramatically easier and more cost-effective to run Cloudera in the cloud. The initial service - available today - targets foundational data engineering workloads and will allow you to quickly and easily spin up Cloudera clusters and run your data pipelines.   Why Cloudera Altus: The platform-as-a-service deployment model eliminates the complexity and overhead associated with cloud and cluster operations for the end user. Instead, users can quickly launch data engineering jobs on-demand and focus on their workloads rather than cluster operations. Altus leverages the trusted, enterprise-grade Cloudera distribution, allowing easy migration of on-prem workloads to and from the cloud with minimal risk. Altus provides interoperable workflows between the data engineering offering and Cloudera’s machine learn

Hello CDH Users, We are pleased to announce the release of Hive ODBC v2.5.22 driver. This release has the following fixes and enhancements: Enhancements & New FeaturesUpdated third-party library dependencies The driver has been updated to incorporate newer versions of the third-party librarydependencies. Resolved IssuesWhen attempting to use the Windows trust store on Windows Server 2016, anaccess violation exception occursDriver converts COALESCE function to less efficient CASE statementConfiguration tool for DSN-less connections, the EnableAuto Reconnect option is disabled by defaultIn some cases, when upgrading the macOS driver from an earlier version to a later one, the installation process fails to update the odbcinst.ini configuration fileSegmentation fault in Driver Manager detection on LinuxIf you use an alias to refer to the Hive server host name while authenticating throughKerberos, the driver fails to connectGetting Started with the Cloudera DriverRead the

Cloudera is pleased to announce the General Availability (GA) of Cloudera Data Science Workbench 1.0. Cloudera Data Science Workbench enables self-service data science and engineering for your secure CDH environment.   Why Cloudera Data Science Workbench: Provide Data Scientists the freedom to use R, Python, or Scala with their favorite libraries and frameworks Directly access data in secure, compliant Hadoop clusters through Apache Spark and Apache Impala Research in a reproducible and collaborative way Run on-premises or in the cloud For more details, see the Cloudera Data Science Workbench release notes.   You can download the Cloudera Data Science Workbench here or register for the final part of our webinar series here. Installation and other documentation for Cloudera Data Science Workbench can be found here.   As always, we are happy to hear your feedback. Please post your comments and suggestions in our community forums.

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.9.2 (CDH 5.9.2, and Cloudera Manager 5.9.2.)This release fixes key bugs and includes the following: CDH fixes for the following issues:FLUME-2812 - Fix semaphore leak causing java.lang.Error: Maximum permit count exceeded in MemoryChannelHBASE-15378 - Scanner cannot handle heartbeat message with no resultsHBASE-15931 - Add log for long-running tasks in AsyncProcess HBASE-16289 AsyncProcess stuck messages need to print region/serverHBASE-16304 - HRegion#RegionScannerImpl#handleFileNotFoundException may lead to deadlock when trying to obtain write lock on updatesLockHDFS-10763 - Open files can leak permanently due to inconsistent lease updateHIVE-12465 - Hive might produce wrong results when (outer) joins are mergedHIVE-12768 - Thread safety: binary sortable serde decimal deserializationHIVE-13539 - HiveHFileOutputFormat searching the wrong directory for HFi

We are pleased to announce the release of Cloudera Director 2.4.0. Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise. Cloudera Director 2.4.0 includes the following new features: Improved support for long-running clusters: Cloudera Director 2.4 will support upgrades, change of services, and configurations from Cloudera Manager 5.11.Spark 2 supportKudu supportFor Azure clusters, support for Azure Data Lake Store (ADLS) via Cloudera Manager and CDH 5.11 deployments.Support for the following Microsoft Azure Clouds and Regions:Azure Germany: Germany Central and Germany NortheastAzure Government: US DoD East and US DoD Central Use the following links to download the product and view the documentation: Download:Download Cloudera DirectorDocumentation:Clouder

Cloudera is pleased to announce that Cloudera Enterprise 5.11 is now generally available (GA). The highlights of this release include lineage support for Apache Spark, Apache Kudu security integration, embedded data discovery for self-service BI, and new cloud capabilities for Microsoft ADLS and Amazon S3. As usual, there are also a number of quality enhancements, bug fixes, and other improvements across the stack. Here is a partial list of what’s included (see the Release Notes for a full list): Core Platform and CloudAmazon S3 Consistency: S3Guard ensures that operations on Amazon S3 are immediately visible to other clients, making it easier to migrate workloads from consistent file systems like HDFS to Amazon S3.Support for Azure Data Lake Store (ADLS): Microsoft introduced ADLS to provide a cost-effective, persistent storage layer for big data applications. With  C5.11, Hive, Spark, and MapReduce can all directly operate on data stored in ADLS, enabling separation of

We are happy to announce Apache Spark 2.1 release 1. You can download the parcel and apply it directly to provisioned clusters without disrupting your currently running Spark workloads.Cloudera Distribution of Apache Spark 2.1 release 1 is compatible with the following CDH versions: CDH 5.7, CDH 5.8, CDH 5.9, CDH 5.10. What's New in Cloudera Distribution of Apache Spark 2.1 Release 1 New direct connector to Kafka that uses the new Kafka consumer API. See Spark 2 Kafka Integration for details. Issues Fixed in Cloudera Distribution of Apache Spark 2.1 - Release 1[SPARK-19554][UI,YARN] Allow SHS URL to be used for tracking in YARN RM.[SPARK-16554][CORE] Automatically Kill Executors and Nodes when they are Blacklisted[SPARK-16654][CORE] Add UI coverage for Application Level Blacklisting[SPARK-8425][CORE] Application Level Blacklisting[SPARK-18117][CORE] Add test for TaskSetBlacklist[SPARK-18949][SQL][BACKPORT-2.1] Add recoverPartitions API to Catalog[SPARK-19459][SQL][BRANCH

Cloudera is happy to announce the availability of parcels and packages for Apache Kudu 1.2.0/ CDH 5.10.1. Apache Kudu 1.2.0/ CDH 5.10.1 is a bug fix release, with no new features or backwards incompatible changes. To upgrade Kudu to 1.2.0/ CDH 5.10.1, see Upgrade Parcels or Upgrade Packages. Issues Fixed in Kudu 1.2.0/ CDH 5.10.1 KUDU-1904 - Fixed a bug where RLE columns with only NULL values would crash on scan.KUDU-1899 - Fixed an issue where tablet servers would crash after inserting an empty string primary key ("").KUDU-1851 - Fixed an issue with the Python client which would crash whenever a TableAlterer is instantiated directly.KUDU-1852 - KuduTableAlterer will no longer crash when given nullptr range bound arguments.KUDU-1821 - Improved warnings when the catalog manager startsFor a complete list of changes, bug fixes, and known issues, see the Kudu 1.2.0/ CDH 5.10.1 Release Notes. As always, your feedback is appreciated. For general Kudu questions, visit the

We are pleased to announce the release of the Cloudera Distribution of Apache Kafka 2.1.1 for CDH 5.  Apache Kafka is a highly scalable, distributed, publish-subscribe messaging system. Apache Kafka 2.1.1 is maintenance release for Kafka 2.1.0. Notable Issues Fixed in Cloudera Distribution of Apache Kafka 2.1.1 KAFKA-3965: MirrorMaker should not commit offset when exception is thrown from producer.sendKAFKA-4158: Reset quota to default value if quota override is deletedKAFKA-2720: Expire group metadata when all offsets have expiredKAFKA-3810: replication of internal topics should not be limited by replica.fetch.max.bytesAll backported fixes can be viewed in the git release notes here or on our website under the Issues fixed section.  We look forward to you trying Kafka 2.1.1. For more information, please use the links below:Install or upgrade KafkaReview the documentationReview the Release Notes As always, we welcome your feedback. Please send your co

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.10.1 (CDH 5.10.1, Cloudera Manager 5.10.1, Cloudera Navigator 2.9.1, and Key Trustee Server 5.10.1). This release fixes key bugs and includes the following: CDH fixes for the following issues:FLUME-2812 - Fix semaphore leak causing java.lang.Error: Maximum permit count exceeded in MemoryChannel.HBASE-15378 - Scanner cannot handle heartbeat message with no results.HBASE-15931 - Add log for long-running tasks in AsyncProcess HBASE-16289 HDFS-11379 - DFSInputStream may infinite loop requesting block locations. AsyncProcess stuck messages need to print region/server.HIVE-12465 - Hive might produce wrong results when (outer) joins are merged.HIVE-15338 - Wrong result from non-vectorized DATEDIFF with scalar parameter of type DATE/TIMESTAMP.IMPALA-4263 - Fix wrong omission of agg/analytic hash exchanges.IMPALA-4282 - Remove max length check for type strings

We are pleased to announce the release of Impala JDBC v2.5.37 and Impala ODBC v2.5.37 drivers. This release has the following fixes and enhancements:  Cloudera JDBC Driver for Impala 2.5.37Enhancements & New Features Specify asynchronous exec poll interval. You can now specify the time in milliseconds between each poll that the driver makes for the query execution status. Specify the number of milliseconds in the Advanced Options dialog box in the Async Exec Poll Interval field, or in the AsyncExecPollInterval configuration option.Support for Impala 2.8. The driver now supports Impala versions 1.0.1 through 2.8.Resolved IssuesUpdate/Delete statements require a semicolon at the end.Conflicting information in documentation regarding CDH. Cloudera ODBC Driver for Impala 2.5.37Enhancements & New FeaturesSpecify asynchronous exec poll interval. You can now specify the time in milliseconds between each poll that the driver makes for the query execution status. Specify

We are pleased to announce the general availability of the Cloudera Connector for Netezza 1.5c5. New Features in Cloudera Connector for Netezza 1.5c5 Fixed import in direct mode with multi-schema by removing unnecessary object owner check.Added fail fast scenario for import with --as-parquetfile in direct mode.Added fail fast scenario for import with --as-avrodatafile in direct mode.For more details on new features and usage of Cloudera Connector for Netezza, see: Release Notes Cloudera Connector for Netezza Version 1.5c5Cloudera Connector for Netezza User Guide, version 1.5c5 As always, we welcome your feedback. Please send your comments and suggestions through our new community forums.  You can also file bugs in the CDH project at issues.cloudera.org.

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.7.6 (CDH 5.7.6, Cloudera Manager 5.7.6, and Cloudera Navigator 2.6.6). This release fixes key bugs and includes the following: CDH fixes for the following issues: HADOOP-13838 - KMSTokenRenewer should close providersHBASE-15430 - Failed taking snapshot - Manifest proto-message too largeHDFS-10271 - Extra bytes are getting released from reservedSpace for appendHDFS-10763 - Open files can leak permanently due to inconsistent lease updateHDFS-11229 - HDFS-11056 failed to close meta fileHIVE-12465 - Hive might produce wrong results when (outer) joins are mergedHIVE-13539 - HiveHFileOutputFormat searching the wrong directory for HFilesHIVE-15338 - Wrong result from non-vectorized DATEDIFF with scalar parameter of type DATE/TIMESTAMPHUE-4466 - [security] Deliver csrftoken cookie with secure bit set if possibleIMPALA-3875 - Thrift threaded server hangs

We are happy to announce Spark 2.0 release 2. You can download the parcel and apply it directly to provisioned clusters without disrupting your currently running Spark workloads. Release 2 addresses a Hive compatibility issue that affects CDH 5.10.1 and higher, CDH 5.9.2 and higher, CDH 5.8.5 and higher, and CDH 5.7.6 and higher. If you are using one of these CDH versions, you must upgrade to the Spark 2.0 release 2 parcel to avoid Spark 2 job failures when using Hive. Release 2 is based on Apache Spark 2.0.2. Issues Fixed in Cloudera Distribution of Apache Spark 2.0 Release 2[SPARK-4563] [CORE] Allow driver to advertise a different network address[SPARK-18993] Unable to build/compile Spark in IntelliJ due to missing Scala deps in spark-tags[SPARK-19314] Do not allow sort before aggregation in Structured Streaming plan[SPARK-18762] Web UI should be http:4040 instead of https:4040[SPARK-18745] java.lang.IndexOutOfBoundsException running query 68 Spark SQL on (100TB)[SPARK

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.8.4 (CDH 5.8.4 and Cloudera Manager 5.8.4).This release fixes key bugs and includes the following: CDH fixes for the following issues: FLUME-2812 - Fix semaphore leak causing java.lang.Error: Maximum permit count exceeded in MemoryChannelHADOOP-12483 - Maintain wrapped SASL ordering for postponed IPC responsesHADOOP-13838 - KMSTokenRenewer should close providersHBASE-15430 - Failed taking snapshot - Manifest proto-message too largeHBASE-17072 - CPU usage starts to climb up to 90-100% when using G1GCHDFS-6962 - ACLs inheritance conflict with umaskmodeHDFS-10763 - Open files can leak permanently due to inconsistent lease updateHIVE-12465 - Hive might produce wrong results when (outer) joins are mergedHIVE-13539 - HiveHFileOutputFormat searching the wrong directory for HFilesHUE-4546 - Auto-strip invalid characters from name field of converted docsIMPALA

We are pleased to announce the release of Cloudera Director 2.3.0. Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise. Cloudera Director 2.3.0 includes the following new features: Support for AWS Spot Blocks. Instances in AWS Spot blocks are guaranteed to remain available during cluster bootstrap, so that the process is resilient to fluctuations in AWS Spot Instance availability.More control of cluster configuration with the introduction of instance-level post-creation scripts, in addition to existing cluster-level post-creation scripts.Updated look and feel of the user interface.AWSNew regions: ap-south-1, ca-central-1, eu-west-1Many new instance typesGCPNew regions: asia-northeast1, us-west1, us-east1AzureNew instance types: DS15_v2, GS4, GS5New OS image: CentO

Cloudera is proud to announce that Cloudera Enterprise 5.10 is now generally available (GA).The highlights of this release include the GA of the new columnar storage engine Apache Kudu, improved cloud performance and cost-optimizations, and cloud-native data governance for Amazon S3.As usual, there are also a number of quality enhancements and bug fixes (learn more about our multi-dimensional hardening/QA process) and other improvements across the stack. Here is a partial list of what’s included (see the Release Notes for a full list): GA of Apache Kudu - Unleash Cloudera’s new storage engine to enable fast analytics on fast changing data with the first generally available release of Kudu. Kudu is purpose-built to enable use cases for time series data, machine data analytics, and online reporting—as part of a complete analytic or operational database.Improved Cloud Performance - Run Cloudera workloads on public cloud infrastructure more efficiently and cost-effectively than ever b

Cloudera takes cluster security very seriously, and provides guidelines for securing CDH environments:   http://www.cloudera.com/documentation/enterprise/latest/topics/sg_edh_overview.html   Security measures become especially important when clusters are exposed to the internet. For example, hackers using network scanning tools are actively looking for WebHDFS ports on clusters, and when they find an open port, they can wreak havoc on the cluster (delete data, steal data, corrupt data).   There are many other services and access considerations that should be protected as well.   It’s imperative that you design clusters in a secure fashion which will not leave the services interfaces exposed to the Internet this way. It’s our strong recommendation that you secure your cluster with kerberos, TLS, proper firewall or proxy access, and use the guidance from our security guide to protect your deployment.   Users affected: All unsecured clusters exposed to the interne

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.9.1 (CDH 5.9.1, and Cloudera Manager 5.9.1, and Cloudera Navigator 2.8.1). This release fixes key bugs and includes the following: CDH fixes for the following issues: HADOOP-12483 - Maintain wrapped SASL ordering for postponed IPC responsesHADOOP-13353 - LdapGroupsMapping getPassward should not return null when IOException throwsHBASE-15430 - Failed taking snapshot - Manifest proto-message too largeHBASE-15856 - Do not cache unresolved addresses for connectionsHDFS-6962 - ACLs inheritance conflict with umaskmodeHDFS-8269 - getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atimeHUE-5028 - Share Oozie workflow with modify permission; however, the user can't edit the shared WFHUE-5163 - Speed up initial page renderingHUE-5041 - Hue export large file to HDFS doesn't work on non-default databas