What's New @ Cloudera

We are pleased to announce the release of Cloudera Enterprise 5.3.8 (CDH 5.3.8, Cloudera Manager 5.3.8, and Cloudera Navigator 2.2.8).  Cloudera Enterprise 5.3.8This release fixes key bugs and includes the following.CDH fixes for the following issues:Spooling Directory Source dies on encountering zero-byte files.Applications fail on NodeManager restart on some Linux distributions because NodeManager container recovery declares AM container as LOST.Fixed a race condition in the ResourceManager's ZooKeeper-based state-store to avoid crashing on duplicate deletes.Allow mlockagent to be used when not starting as root.java.lang.IndexOutOfBoundsException when union all with if function.For a full list of upstream JIRAs fixed in CDH 5.3.8, see the issues fixed section of the Release Notes.Cloudera Manager fixes for the following issues:Rolling Restart fails when inheriting inappropriate JVM properties.Exception thrown when viewing host configuration change.  For full list of issues

We are pleased to announce the release of Impala JDBC v2.5.28 and Impala ODBC v2.5.30. This release has the following fixes and enhancements: JDBC:Features addedNested data types (ARRAY, MAP, and STRUCT) now supported.Issues resolvedWhen using executeUpdate in a prepared statement to insert more than 2 rows, the driver returns an error.ODBC:Features addedNested data types (ARRAY, MAP, and STRUCT) now supported.Added MANIFEST file for the OpenSSL library.Data encryption and integrity support implemented for the SASL SSPI plug-in.Issues resolvedSQLCancel was slow to execute and did not cancel the query in the Impala server. Now, SQLCancel works as expected. Please see Release Notes for ODBC and JDBC for details on all of the fixes and enhancements. Getting started with the Cloudera Drivers:Read the Cloudera JDBC 2.5.28  Driver for Impala Release Notes and Installation Guide.Read the Cloudera ODBC 2.5.30  Driver for Impala Release Notes and Installation Guide.Download the c

We are pleased to announce the release of Cloudera Director 1.5.1.   Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise.   Cloudera Director 1.5.1 has bug fixes and and product improvements, as outlined in Cloudera Director 1.5.1 release notes.   Links to the product and documentation are listed below: Download Cloudera Director from http://www.cloudera.com/content/support/en/downloads.html View the documentation: Cloudera Director Release Notes Getting Started Installing Cloudera Director Client Installing Cloudera Director Server To learn more about the Director SPI and plug-in implementations: Cloudera Director Service Provider Interface (Cloudera Director SPI) Cloud Provider Technology Compatibility Kit (Cloudera Director SPI TCK)

Dear CDH, Cloudera Manager, Impala, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.4.7 (CDH 5.4.7, Cloudera Manager 5.4.7, and Cloudera Navigator 2.3.7).   Cloudera Enterprise 5.4.7 This release fixes key bugs and includes the following. CDH fixes for the following issues: The Spooling directory source dies when encountering zero-byte files. The VolumeScanner thread exits with an exception if there is no block pool to be scanned but there are suspicious blocks. ArrayIndexOutOfBoundsException in CellComparator#getMinimumMidpointArray. Lateral view on top of a view throws a RuntimeException. java.lang.IndexOutOfBoundsException when union all with if function. For a full list of upstream JIRAs fixed in CDH 5.4.7, see the issues fixed section of the Release Notes.   Cloudera Manager fixes for the following issues: The ZooKeeper jute.maxbuffer property is emitted into zoo.cfg instead of in the JVM arguments. Usi

Dear CDH, Cloudera Manager, Impala, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.4.5 (CDH 5.4.5, Cloudera Manager 5.4.5, and Cloudera Navigator 2.3.5).   Cloudera Enterprise 5.4.5 This release fixes key bugs and includes the following. CDH fixes for the following issues: Predicate push-down does not work with Parquet when partitions are in the expression. Impala DML/DDL operations corrupt table metadata leading to Hive query failures. Reserved RBW space is not released if the client terminates while writing a block. NullPointerException on queries where MapReduce is not involved on tables with partitions. Parallel move task optimization causes a race condition. ResultSerialization and MutationSerialization can throw InvalidProtocolBufferException when serializing a cell larger than 64 MB. For a full list of upstream JIRAs fixed in CDH 5.4.5, see the issues fixed section of the Release Notes. Cloudera Manager fixes

We are pleased to announce the release of Cloudera Director 1.5.0.| Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise. Cloudera Director 1.5.0 includes the following new features: Cloudera Director now supports multiple cloud providers through an open-source plugin interface: Cloudera Director Service Provider Interface (Cloudera Director SPI). Google Cloud Platform is now supported out-of-the-box through an open-source implementation of the Cloudera Director SPI: Cloudera Director Google Plugin. Advanced configuration settings are now available in the Cloudera Director UI. These include CDH cluster and Cloudera Manager configurations, and management of database servers set up by Cloudera Director. Users can now specify custom scripts to run after cluster creation

Dear CDH, Cloudera Manager, Impala, and Cloudera Navigator users,   We are pleased to announce the release of Cloudera Enterprise 5.3.6 (CDH 5.3.6, Cloudera Manager 5.3.6, and Cloudera Navigator 2.2.6).     This release fixes key bugs and includes the following. CDH fixes for the following issues: NameNode Failover during HA upgrade can cause DataNode to finalize upgrade HBase grant at specific column level does not work for Groups ZooKeeper znodes are bootstrapped with insecure ACLs in a secure configuration Incorrect results for n-way join when join expressions are not in same order across joins Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume For a full list of upstream JIRAs fixed in CDH 5.3.6, see the issues fixed section of the Release Notes. Cloudera Manager fixes for the following issues: Cloudera Manager reports the wrong value for Impala bytes read from cache Cancel Impala Query

Dear CDH Users,     We are pleased to announce the release of Cloudera Distribution of Apache Kafka 1.3.1 for CDH 5.  Apache Kafka is a distributed publish-subscribe messaging system.  This is a maintenance release that fixes key issues, including:   KAFKA-972 - MetadataRequest returns stale list of brokers KAFKA-1367 - Broker topic metadata not kept in sync with ZooKeeper KAFKA-1867 - liveBroker list not updated on a cluster with no topics KAFKA-2308 - New producer + Snappy face un-compression errors after broker restart KAFKA-2317 - De-register isrChangeNotificationListener on controller resignation KAFKA-2337 - Verify that metric names will not collide when creating new topics We look forward to you trying it out, using the information below: Install or upgrade Kafka Review the Documentation Review the Release Notes   As always, we welcome your feedback. Please send your comments and suggestions through our community forums. Y

Dear CDH Users,   We are pleased to announce the release of the Cloudera Connector for Netezza version 1.3. This release has the following new feature:   A new --schema argument is now supported. This argument defines the schema that Sqoop uses for both import and export. The schema is used for metadata queries, and for record extraction and loading (available in Netezza 7.0.3 or later). This connector release is available for CDH 5 only. We look forward to you trying it out, using the information below: Download the Connector Review the Release Notes Review the User Guide As always, we are happy to hear your feedback. Please send your comments and suggestions through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org.

Hello CDH and Impala Users,   We are pleased to announce the release of Impala ODBC v2.5.29, Impala JDBC v2.5.24 and Hive JDBC v2.5.15.   Impala ODBC 2.5.29 contains bug fixes and the following enhancements: Support for calling SQLGetData on columns that come before a bound column Updated support for SQL_C_TINYINT Improved error messaging for authentication issues Support for saving passwords as encrypted strings in the Windows Registry Supports the use of "_HOST" as a special value for the Host FQDN option For a full list of fixes in Impala ODBC v2.1.29, see the issues fixed section of the Release Notes.   Impala JDBC 2.5.24 is a bugfix only release. For a full list of fixes in Impala JDBC v2.1.24, see the issues fixed section of the Release Notes.   Hive JDBC v2.5.15 contains numerous bug fixes and the following enhancements: Support for case-insensitive treatment of catalog names, schema names, table names, and column names in catalog fun

Dear CDH, Cloudera Manager, Impala, and Cloudera Navigator users,   We are pleased to announce the release of Cloudera Enterprise 5.2.6 (CDH 5.2.6, Cloudera Manager 5.2.6, and Cloudera Navigator 2.1.6).   Cloudera Enterprise 5.2.6 This release fixes key bugs and includes the following.   CDH fixes for the following issues: YARN rolling upgrade issue ArrayIndexOutOfBoundsException in NN when starting a DN after a storageID change Error that overestimates the amount of space to reserve for open files and can result in storage being incorrectly marked as full. For a full list of upstream JIRAs fixed in CDH 5.2.6, see the issues fixed section of the Release Notes.   Cloudera Manager fixes for the following issues: Cloudera Manager Agent may become slow or get stuck when responding to commands and when sending heartbeats to Cloudera Manager Server. Cloudera Manager reports the wrong value for Impala bytes read from cache. Cloudera Manager use

Dear CDH, Cloudera Manager, Cloudera Impala, and Cloudera Navigator users,   CDH 5.4.4 fixes the following critical bugs:   Hue will not start when using SSL HiveServer2 LDAP Provider May Allow Authentication with Blank Password We look forward to you trying it out, using the information below: Download Cloudera Enterprise View the documentation As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org.

Dear CDH, Cloudera Manager, Impala, and Cloudera Navigator users,   We are pleased to announce the release of Cloudera Enterprise 5.4.3 (CDH 5.4.3, Cloudera Manager 5.4.3, and Cloudera Navigator 2.3.3).   Cloudera Enterprise 5.4.3This release fixes key bugs and includes the following. CDH fixes for the following issues: Multiple YARN rolling upgrade issues  Potential data loss in the HDFS DataNode Incremental block report for HDFS significantly slows down NameNode startup For a full list of upstream JIRAs fixed in CDH 5.4.3, see the Known issues fixed section of the Release Notes. Cloudera Manager fixes for the following issues: Upgrade Wizard fails during CDH 5.4.1/Cloudera Manager 5.4.1 to CDH 5.4.3/Cloudera Manager 5.4.3 upgrade Performance issues when changing configurations on HDFS Cloudera Manager reports the wrong value for Impala bytes read from cache Multiple cross-site scripting vulnerabilities For full list of issues fixed in Cloudera Manager 5.4.3, see t

Dear CDH users,   This release fixes an issue that caused rolling upgrades to CDH 5.3.4 to fail.   We look forward to you trying it out, using the information below:   Download CDH 5.3.5 from the download page View the documentation: CDH 5.3.x Release Notes As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org.

Dear CDH, Cloudera Manager, Impala, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.3.4 (CDH 5.3.4, Cloudera Manager 5.3.4, and Cloudera Navigator 2.2.4).     Cloudera Enterprise 5.3.4 This release fixes key bugs and includes the following. CDH Multiple fixes for HDFS rolling upgrades Multiple YARN fair scheduler fixes Impala performance and query fixes HS2 no longer leaks open file descriptors when using UDFs For full list of upstream JIRAs fixed in CDH 5.3.4, see the Known issues fixed section of the Release Notes. Cloudera Manager fixes for the following issues: Slowness observed when accessing replication page. Cloudera Manager overwrites the krb5.conf file after disabling management by Cloudera Manager. Header injection vulnerability in internal logging call. Graph for "Total Containers Running Across NodeManagers" displays incorrect high values. We look forward to you trying it out, using the i

Hello CDH and Impala Users, We are pleased to announce the release of Hive ODBC v2.5.15 and Impala JDBC v2.5.28. Hive ODBC v2.5.15 contains bug fixes and the following enhancements: Two-way SSL verification Custom HTTP headers SSL and Thrift transport protocol now configured separately from authentication Support for saving passwords as encrypted strings in the Windows Registry Impala ODBC 2.5.28 contains bug fixes and the following enhancements: Driver configuration option for enabling or disabling connections to SSL-enabled servers that use self-signed certificates Ability to configure SSL separately from authentication Option for configuring SASL when using username and password authentication   These drivers work for versions of HiveServer2 in CDH 4.1 or higher and Impala 1.0 or higher. Previous drivers continue to work on the latest versions of HiveServer2 and Impala.   Getting started with the Cloudera drivers: Read the Cloudera ODBC 2.5 driver for

Hello CDH and Impala Users,   We are pleased to announce the release of Hive JDBC v2.5.12 and Impala JDBC v2.5.22. Hive JDBC v2.5.12 allows you to configure SSL separately from authentication and fixes the following issues: The getColumns() function returns an empty result set if it uses both schema and table restrictions, and the table restriction contains an escaped wildcard character. Query translation does not use the database context specified by a USE <database> query. The driver does not report an error when a MapReduce job fails on the server side during query execution.   Impala JDBC 2.5.22 fixes the following issues: The getColumns() function returns an empty result set if it uses both schema and table restrictions, and the table restriction contains an escaped wildcard character. Precision and scale for DECIMAL columns are reported incorrectly.   Query translation does not use the database context specified by a USE <database> query

Dear CDH, Cloudera Manager, Impala, and Cloudera Navigator users,   This release fixes an issue that caused upgrades to CDH 5.4.1 to potentially fail.     We look forward to you trying it out, using the information below: Download Cloudera Enterprise from: http://www.cloudera.com/content/support/en/downloads.html   View the documentation: CDH 5 Release Notes Cloudera Manager Release Notes Cloudera Navigator Release Notes   As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external JIRA projects on issues.cloudera.org.

We are pleased to announce the release of Cloudera Director 1.1.3. Cloudera Director is a simple, reliable, enterprise-grade way to deploy, scale, and manage Hadoop in the cloud, initially for Amazon Web Services (AWS). It is free to download and use, and supported for Cloudera Enterprise customers. This release fixes key bugs and includes the following: Uses lazy_itable_init when preparing ephemeral drives to speed up instance preparation.  This is useful for instances that have many large ephemeral drives, such as AWS D2 instances. Adds support for GUID Partition Table (GPT) root disk partitions. Allows custom AWS Identity and Access Management (IAM) endpoints. Changes the default Cloudera Manager and CDH versions to 5.3.3 instead of the latest version to ensure that compatible Cloudera Manager and CDH versions are used over the lifetime of Cloudera Manager. Corrects a typo for the d2.4xlarge instance type in virtualizationmappings.properties. No longer inadvertently

Dear CDH, Cloudera Manager, Impala, and Cloudera Navigator users,   We are pleased to announce the release of Cloudera Enterprise 5.4.1 (CDH 5.4.1, Cloudera Manager 5.4.1, and Cloudera Navigator 2.3.1).     Cloudera Enterprise 5.4.1 This release fixes key bugs and includes the following. CDH Optimization to avoid opening too many file descriptors during encryption DistCp fix to handle copying to root directory HDFS fixes for: Edit log corruption when append fails with a quota violation. Rolling upgrade restoring blocks from trash multiple times, which could cause. significant and unnecessary block churn. NameNode failover during high availability (HA) upgrade, which can cause the DataNode to finalize the upgrade. Null pointer exception (NPE) when NameNode transitions to active during startup. Incorrectly allowing HA NameNode to start in secure mode with HTTP wildcard address configured. Handling cache checks when encryption is enabled. YARN fi

Hi CDH, Cloudera Manager, and Cloudera Navigator users,   We’re pleased to announce the release of Cloudera Enterprise 5.4 (comprising CDH 5.4, Cloudera Manager 5.4, and Cloudera Navigator 2.3). Cloudera Enterprise 5.4 (Release Notes) reflects critical investments in a production-ready customer experience through  governance, security, performance and deployment flexibility in cloud environments. It also includes support for a significant number of updated open standard components–including Apache Spark 1.3, Impala 2.2, and  Apache HBase 1.0 (as well as unsupported beta releases of Hive-on-Spark data processing and OpenStack deployments). Recently Cloudera made the upgrade process considerably easier via an improved CDH upgrade wizard; see details about that wizard here and best practices here. (Note: Due to metadata format changes in Apache Hadoop 2.6, upgrading to CDH 5.4 and later from any earlier release requires an HDFS metadata upgrade, as well. Note also that for

Dear Cloudera Users,   We are pleased to announce the general availability of the Cloudera Connector Powered by Teradata 1.4. This release has many updates, including: Added support for JDK 8 (CDH 5 only). Added the --error-database option. Added the ability to specify format of Date, Time and Timestamp types when importing into CSV. Import method split.by.amp now supports views. Upgraded Teradata connector for Hadoop to version 1.3.4. The Connector is released in editions compatible with CDH 4 and CDH 5. See the download page for more details. For more details on new features and usage of Cloudera Connector Powered by Teradata, see: Release Notes Cloudera Connector Powered by Teradata version 1.4 Cloudera Connector Powered by Teradata User Guide, version 1.4 As always, we welcome your feedback. Please send your comments and suggestions through our new community forums.  You can also file bugs in the CDH project at issues.cloudera.org.

Dear CDH, Cloudera Manager, Impala, and Cloudera Navigator users,   We are pleased to announce the release of Cloudera Enterprise 5.1.5 (CDH 5.1.5, Cloudera Manager 5.1.5, and Cloudera Navigator 2.0.5) and Cloudera Enterprise 5.2.5 (CDH 5.2.5, Cloudera Manager 5.2.5, and Cloudera Navigator 2.1.5).   Cloudera Enterprise 5.1.5 This release fixes key bugs and includes the following. CDH Replacing DataNode disks or manually changing the Storage IDs of volumes in a cluster may result in data loss. Fixed as part of HDFS-7575, HDFS-7596, HDFS-7960 and HDFS-7208. Fixes to the dfsadmin command-line option and the inclusion of a command to manually trigger full block reports from a DataNode. Improved recovery speed after service or host failure, and fixes to handle race conditions with HDFS integration in Search. Bug fixes in YARN for handling display of clock skew, consistent time zone on the web UI, and ResourceManager high availability. Fix to address LDAP authentication,

We are pleased to announce the release of Cloudera Director 1.1.2.   Cloudera Director is a simple, reliable, enterprise-grade way to deploy, scale, and manage Hadoop in the cloud (initially for AWS). It’s free to download and use, and supported by default for Cloudera Enterprise customers. This release is focused on key bugs fixes including: Added support for d2 and c4 AWS instance types Restored the ability to modify a cluster that had an empty initial list of parcel repositories and was created with Director 1.0 Fixed parcel version validation when using an HTTP proxy We look forward to you trying it out, using the information below: Download Director from: http://www.cloudera.com/content/support/en/downloads.html View the documentation: Cloudera Director Release Notes Getting Started Installing Cloudera Director Client Installing Cloudera Director Server As always, we are happy to hear your feedback. Please send your comments and suggestions to the use

Dear CDH, Cloudera Manager, and Impala users,   We are pleased to announce the release of Cloudera Enterprise 5.0.6 (CDH 5.0.6 and Cloudera Manager 5.0.6)   This release fixes key bugs and includes the following. CDH Replacing DataNode disks or manually changing the Storage IDs of volumes in a cluster may result in data loss. Fixed as part of HDFS-7575, HDFS-7596, HDFS-7960 and HDFS-7208. Improved recovery speed after service or host failure, and fixes to handle race conditions with HDFS integration in Search. Bug fixes in YARN for handling display of clock skew and consistent time zone on the web UI. Bug fixes in Hive, including integration of Impala with HBase through Oozie. For full list of upstream JIRAs fixed in CDH 5.0.6, please see the Known issues fixed section of the Release Notes. Cloudera Manager Starting new ZooKeeper servers shortly after adding them no longer causes ZooKeeper data loss when the number of new servers exceeds the number of old