We are pleased to announce the release of the Cloudera Distribution of Apache Kafka 2.0.2 for CDH 5.  Apache Kafka is a highly scalable, distributed, publish-subscribe messaging system. Apache Kafka 2.0.2 is maintenance release for Kafka 2.0. Notable Issues Fixed in Cloudera Distribution of Apache Kafka 2.0.2: KAFKA-3495: NetworkClient.blocking Send And Receive should rely on requestTimeout. KAFKA-2998: Log warnings when client is disconnected from bootstrap brokers. KAFKA-3488: Avoid failing of unsent requests in consumer where possible. KAFKA-3528: Handle wakeups while rebalancing more gracefully. KAFKA-3594: After calling MemoryRecords.close() method, hasRoomFor() method should return false. KAFKA-3602: Rename RecordAccumulator dequeFor() and ensure proper usage. KAFKA-3789: Upgrade Snappy to fix Snappy decompression errors. KAFKA-3830: getTGT() debug logging exposes confidential information. KAFKA-3840: Allow clients default OS buffer sizes. KAFKA-3691: Confusing logging during metadata update timeout. KAFKA-3810: Replication of internal topics should not be limited by replica.fetch.max.bytes. KAFKA-3854: Fix issues with new consumer's subsequent regex (pattern) subscriptions. All backported fixes can be viewed in the git release notes here. We look forward to you trying Kafka 2.0.2! For more information, please use the links below: Install or upgrade Kafka Review the documentation Review the Release Notes   As always, we welcome your feedback. Please send your comments and suggestions through our community forums.     ... View more

We are pleased to announce the release of Cloudera ODBC 2.5.34 Driver for Impala. This release has the following fixes and enhancements:   Enhancements and New Features Support for queries that use the EXISTS syntax. Additional SQL Column optimization for catalog functions that include schema restrictions.   Resolved Issues Query fails if the column name contains a character from the Latin-1 Supplement character set. Warning dialog when non-Admin user clicks OK on the DSN dialog should be suppressed. Driver reports statement length limit as 16 KB.   Getting Started with the Cloudera Drivers Read the Cloudera ODBC 2.5.34 Driver for Impala Release Notes and Installation Guide. Download the connector from the Cloudera Connectors page.   As always, we welcome your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external JIRA projects on issues.cloudera.org.   ... View more

We are pleased to announce the release of Cloudera Director 2.1.0. Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise. Cloudera Director 2.1.0 includes the following new features:   Usage-based billing for Cloudera services Support for running Cloudera Manager and CDH clusters on Microsoft Azure Support for deploying Cloudera Manager and CDH on a different cloud provider or region than Cloudera Director with a simple network setup Supports Cloudera 5.7 out of the box Various bug fixes and improvements Links to the product and documentation are listed below: Download Cloudera Director from: http://www.cloudera.com/content/support/en/downloads.html View the documentation: Cloudera Director Release Notes Getting Started Installing Cloudera Director Client Installing Cloudera Director Server   As always, we are happy to hear your feedback. Please post your questions and comments on our community forum.   ... View more

  We are pleased to announce the release of Impala JDBC v2.5.32 driver. This release has the following fixes and enhancements: Enhancements and new features: Support added for optimizing driver performance for INSERT statements. You can use the new Optimized Insert connection property to optimize driver performance when executing INSERT statements. This optimization is enabled by default. Support added for configuring the Thrift transport protocol. You can use the new transport Mode connection property to specify the transport protocol that the driver uses in the Thrift layer. The default AuthMech value now depends on the transport Mode setting. For more information, see the Cloudera JDBC Driver for Impala Installation and Configuration Guide. Implemented server version checking for the getTypeInfo() method. When you call the getTypeInfo() method, the driver now checks the version of the Impala server that you are connecting to and only returns data types that are supported by that Impala version. Issues resolved: When using a single statement to insert multiple rows, an OutOfMemory error occurs. In some cases, when assertions are enabled and an error occurs, the driver returns an assertion error instead of the appropriate error. The driver incorrectly reports the buffer length and octet length of CHAR and VARCHAR data as 0. In some cases, the driver returns a negative value for column size. The data type names returned by the getTypeInfo() method are different from the data type names used in Impala.   See Release Notes for JDBC for details on all of the fixes. Getting Started with the Cloudera Drivers Read the Cloudera JDBC 2.5.32 Driver for Impala Release Notes and Installation Guide. Download the connector from the Cloudera Connectors page.   As always, we welcome your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external JIRA projects on issues.cloudera.org. ... View more

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.7.1 (CDH 5.7.1, Cloudera Manager 5.7.1, and Cloudera Navigator 2.6.1).   This release fixes key bugs and includes the following: CDH fixes for the following issues: HBASE-15234 - The ReplicationCleaner process can abort if its connection to ZooKeeper is inconsistent HDFS-8211 - DataNode UUID is always null in the JMX counter YARN-4414 - NodeManager connection errors are retried at multiple levels YARN-4629 - Distributed shell breaks under strong security HIVE-13401 - Kerberized HS2 with LDAP auth enabled fails Kerberos/delegation token authentication HIVE-13530 - Hive on Spark throws Kryo exception in some cases IMPALA-3133 - Wrong privileges after a REVOKE ALL ON SERVER statement IMPALA-3139 - Fix drop table statement to not drop views and vice versa SENTRY-1190 - IMPORT TABLE silently fails if Sentry is enabled For a full list of upstream JIRAs fixed in CDH 5.7.1, see the issues fixed section of the Release Notes. Cloudera Manager fixes for the following issues: Cloudera Manager HDFS usage reports do not include Inode references Authentication errors occur due to missing SAML metadata HiveServer2 Web UI did not use SSL when Kerberos was enabled Impala does not throw null pointer exception when memory limit is not set Unable to stop Cloudera Manager Agent on SLES 11 For a full list of issues fixed in Cloudera Manager 5.7.1, see the issues fixed section of the Release Notes. Cloudera Navigator fixes for the following issues: Logical Lineage returns deleted entities even if the Hide deleted entities option is selected Physical relationships unexpectedly shown in lineage ClassCastException thrown when a custom entity is encountered For a full list of issues fixed in Cloudera Navigator 2.6.1, see the issues fixed section of the Release Notes. We look forward to you trying it, using the information below: Download Cloudera Enterprise View the documentation   As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org.   ... View more

We are pleased to announce the release of Cloudera Enterprise 5.5.4 (CDH 5.5.4, Cloudera Manager 5.5.4, and Cloudera Navigator 2.4.4).   This release fixes key bugs and includes the following: CDH fixes for the following issues: FLUME-2886 - Optional Channels can cause OOMs HADOOP-12718 - Incorrect error message by fs -put local dir without permission HDFS-9406 - FSImage may get corrupted after deleting snapshot HDFS-9514 - TestDistributedFileSystem.testDFSClientPeerWriteTimeout failing; exception being swallowed HDFS-9701 - DN may deadlock when hot-swapping under load HBASE-15234 The Replication Cleaner process can abort if its connection to ZooKeeper is inconsistent MAPREDUCE-6460 - TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails MAPREDUCE-6528 - Memory leak for HistoryFileManager.getJobSummary() YARN-3446 - FairScheduler headroom calculation should exclude nodes in the blacklist HBASE-15019 - Replication stuck when HDFS is restarted HIVE-10303 - HIVE-9471 broke forward compatibility of ORC files HIVE-13065 - Hive throws NPE when writing map type data to an HBase backed table HIVE-13243 - Hive drop table on encryption zone fails for external tables HIVE-13401 - Kerberized HS2 with LDAP auth enabled fails Kerberos/delegation token authentication HUE-3132 - [core] Fix Sync LDAP users and groups for anonymous binds IMPALA-2592 - DataStreamSender::Channel::CloseInternal() does not close the channel on an error IMPALA-2599 - Pseudo-random sleep before acquiring Kerberos ticket possibly not really pseudo-random IMPALA-2711 - Fix memory leak in Rand() IMPALA-2719 - test_parquet_max_page_header fails on Isilon IMPALA-3095 - Add configurable whitelist of authorized internal principals KITE-1114 - Kite CLI json-import HDFS temp file path not multiuser safe, fix missing license header OOZIE-2419 - HBase credentials are not correctly proxied SENTRY-1044 - Tables with non-HDFS locations break HMS startup SOLR-8215 - Only active replicas should handle incoming requests against a collection SPARK-10914 - UnsafeRow serialization breaks when two machines have different Oops size SPARK-12617 - [PYSPARK] Move Py4jCallbackConnectionCleaner to Streaming, clean up the leak sockets of Py4J SQOOP-2847 - Sqoop --incremental + missing parent --target-dir reports success with no data For a full list of upstream JIRAs fixed in CDH 5.5.4, see the issues fixed section of the Release Notes. Cloudera Manager fixes for the following issues: Kafka MirrorMaker would not start when Kafka was installed using packages. This occurred because KAFKA_HOME was not set to the correct default when starting MirrorMaker. This issue affected Cloudera Manager 5.4.0 and higher with Kafka 1.4.0 and higher. When Sentry Sync is enabled, new Hive tables failed to replicate. Replication now occurs as expected. In previous releases, Cloudera Manager SAML metadata was missing an alias and signature. As a result, errors sometimes occur when logging out. In some cases, provisioning of a cluster may fail at the start of the process. This does not happen in all cases and is mainly noticed on RHEL 6 and especially when some hosts are reporting bad health. Parcel activation and first run command now complete as expected, even when some hosts report bad health. For a full list of issues fixed in Cloudera Manager 5.5.4, see the issues fixed section of the Release Notes. Issues Fixed in Cloudera Navigator 2.4.4 Lineage does not render on MacOS using Chrome 48.0.2564.109. After upgrade to 2.4.x, only entries related to deleted files are visible. For full list of issues fixed in Cloudera Navigator 2.4.4, see the issues fixed section of the Release Notes.   Issues Fixed in Key Trustee KMS 5.5.4 If the Key Trustee KMS 5.5.0 parcel is deactivated, any existing GPG keys are also deactivated. If the parcel is then reactivated, new GPG keys (used to create an authenticated and private communication channel with the Key Trustee Server) are generated. The existing GPG keys that were in use before the deactivation are not lost; however, they become inactive. If remedial action is not taken before deactivation, this can result in a loss of access to HDFS Encryption Zone keys generated with the older set of GPG keys. This in turn leads to loss of access to all data in all encryption zones. As long as the Key Trustee KMS parcel directory is not deleted, access can be restored. Assistance from Cloudera Support may be required. See TSB 2016-121 for more information (requires login to the Cloudera Support Portal). The UNDELETE and PURGE ACL entries were being read from kms-site.xml instead of kms-acls.xml. For full list of issues fixed in Key Trustee KMS 5.5.4, see the issues fixed section of the Release Notes.   We look forward to you trying it, using the information below: Download Cloudera Enterprise View the documentation   As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org. ... View more

We are pleased to announce the release of Cloudera Enterprise 5.6.1 (CDH 5.6.1, Cloudera Manager 5.6.1, and Cloudera Navigator 2.5.1).   This release fixes key bugs and includes the following: CDH fixes for the following issues: FSImage may get corrupted after deleting snapshot The ReplicationCleaner process can abort if its connection to ZooKeeper is inconsistent FLUME-2886 - Optional Channels can cause OOMs HADOOP-12604 - Exception may be swallowed in KMSClientProvider HADOOP-12718 - Incorrect error message by fs -put local dir without permission MAPREDUCE-6528 - Memory leak for HistoryFileManager.getJobSummary() HIVE-12790 - Metastore connection leaks in HiveServer2 IMPALA-2711 - Fix memory leak in Rand() IMPALA-2820 - Support unquoted keywords as struct-field names SENTRY-835 - Drop table leaves a connection open when using metastorelistener For a full list of upstream JIRAs fixed in CDH 5.6.1, see the issues fixed section of the Release Notes. Cloudera Manager fixes for the following issues: Scheme and location not filled in consistently during Hive replication import Cloudera Manager HDFS usage reports do not include Inode references Kafka MirrorMaker unable to start due to KAFKA_HOME not being set kt_renewer not automatically created when Hue is added to a Kerberos-enabled cluster Authentication errors occur due to missing SAML metadata For a full list of issues fixed in Cloudera Manager 5.6.1, see the issues fixed section of the Release Notes. We look forward to you trying it, using the information below: Download Cloudera Enterprise View the documentation   As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org. ... View more

We are pleased to announce the release of Hive ODBC v2.5.19 and Impala ODBC v2.5.33 drivers. This release has the following fixes and enhancements:   Hive ODBC v2.5.19 Enhancements and New Features -Support added for Red Hat Enterprise Linux (RHEL) 7 and CentOS 7 -Updated authentication support. For consistency with Microsoft Azure HDInsight updates, the Windows Azure -HDInsight Emulator Authentication mechanism (AuthMech=5) is now deprecated. The driver no longer supports connections to Hive server instances that run on Windows Azure HDInsight Emulator.   Resolved Issues -When attempting to connect to the server using a non-Windows version of the driver with Service Discovery Mode enabled, the client stops working. -When executing a parameterized INSERT statement on a DATE, DECIMAL, or TIMESTAMP column, an error occurs. -When executing a query that contains a large number of filters using OR operators, an error occurs. -When executing a query that uses regular expressions in the fields, an error occurs. Before, this problem occurred when querying Hive servers that are configured to not allow quoted identifiers (the hive.support.quoted.identifiers parameter is set to none), because these servers cannot parse the backquotes (`) used for regular expressions. Driver truncates passwords that exceed the maximum character length of the Password Field in the Cloudera Hive ODBC Driver DSN Setup dialog box. -The maximum character length of the Password field has been increased to 5000 characters. -In some cases, the driver stops working intermittently. Before, this problem occurred due to improper timing and sequence of operations between the execution context and the fetch thread. -In some cases, after being used in Tableau, the driver caches sessions and then tries to reuse them the next time Tableau is started. -When executing a parameterized INSERT statement on a BINARY column, an error occurs. Impala ODBC v2.5.33 Enhancements and New Features -Support added for Red Hat Enterprise Linux (RHEL) 7 and CentOS 7 -Improved implementation of SQLCancel -Optimized metadata retrieval. The driver now retrieves schemas and table names using the SHOW TABLES query instead of the GetTables Thrift API call. Resolved Issues -The Use Only SSPI configuration option (the User Only SSPI property) does not work correctly. [12888] Driver does not push down restrictions for catalog functions (SQLTables SQLColumns), in some cases causing a large number of GET_TABLES requests to occur. When connecting to Impala 1.1 and executing a query that contains a NULL literal in the SELECT list, the driver terminates unexpectedly. Getting Started with the Cloudera Drivers -Download the Cloudera ODBC 2.5.19 Driver for Hive, and read the Release Notes and Installation Guide. -Download the Cloudera ODBC 2.5.33 Driver for Impala, and read the Release Notes and Installation Guide. As always, we welcome your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org. ... View more

We are pleased to announce the release of Cloudera Enterprise 5.3.10 (CDH 5.3.10, Cloudera Manager 5.3.10, and Cloudera Navigator 2.2.10).   This release fixes key bugs and includes the following: CDH fixes for the following issues: HDFS-9406 - When deleting a snapshot that contains the last record of a given INode, the fsimage may become corrupt because the create list of the snapshot diff in the previous snapshot and the child list of the parent INodeDirectory are not cleaned. HADOOP-12464 - Interrupted client may try to fail over and retry HADOOP-12604 - Exception may be swallowed in KMSClientProvider HDFS-8211 - DataNode UUID is always null in the JMX counter HDFS-9092 - NFS silently drops overlapping write requests and causes data copying to fail HDFS-9358 - TestNodeCount#testNodeCount timed out MAPREDUCE-6302 - Incorrect headroom can lead to a deadlock between map and reduce allocations YARN-3464 - Race condition in LocalizerRunner kills localizer before localizing all resources YARN-4235 - FairScheduler PrimaryGroup does not handle empty groups returned for a user HBASE-11394 - AmendReplication can have data loss if peer ID contains hyphen HBASE-12336 - RegionServer failed to shutdown for NodeFailoverWorker thread HBASE-13437 - ThriftServer leaks ZooKeeper connections HBASE-15019 - Replication stuck when HDFS is restarted HIVE-8115 - Hive select query hangs when fields contain map IMPALA-1702 - "invalidate metadata" can cause duplicate TableIds (issue not entirely fixed, but now fails gracefully) IMPALA-3095 - Allow additional Kerberos users to be authorized to access internal APIs SENTRY-835 - Drop table leaves a connection open when using metastorelistener SENTRY-957 - Exceptions in MetastoreCacheInitializer should probably not prevent HMS from starting up For a full list of upstream JIRAs fixed in CDH 5.3.10, see the issues fixed section of the Release Notes. Cloudera Manager fixes for the following issues: Scheme and location not filled in consistently during Hive replication import. In previous releases, Hive replication import phase did not consistently fill in scheme and location information. This information is now filled in as expected. YARN jobs fail after enabling Kerberos authentication or selecting Always Use Container Executor. After Kerberos security is enabled on a cluster or Always Use Container Executor is selected, YARN jobs failed. This occurred because the contents of any previously existing YARN User Cache directory could not be overridden after security was enabled. YARN jobs now complete as expected after a change in Kerberos security or usage of Container Executor. For full list of issues fixed in Cloudera Manager 5.3.10, see the issues fixed section of the Release Notes. We look forward to you trying it, using the information below: Download Cloudera Enterprise View the documentation   As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org. ... View more