We are pleased to announce the release of Impala JDBC v2.5.31 driver. This release has the following fixes and enhancements: Issues resolved: When translating a query that contains an implicit join, the driver disables the implicit join by using CROSS JOIN syntax. If an invalid server-side property is specified, the driver fails to connect. The driver now posts a warning for the invalid server-side property and continues the connection process. When handling multiple connections at the same time, in some cases the driver returns a conversion error when attempting to convert INT data into Long data. The driver no longer removes hints from queries. To preserve hints, the driver does not translate queries that contain hints even when it is configured to work in translation mode (UseNativeQuery=0). Driver returns error when casting data to type TIMESTAMP. When the Driver.getPropertyInfo() method is called, the driver returns a null pointer exception. Driver casts NULL values to type STRING regardless of the return types specified in the searched CASE expression in the query.   See Release Notes for JDBC for details on all of the fixes. Getting Started with the Cloudera Drivers Read the Cloudera JDBC 2.5.31 Driver for Impala Release Notes and Installation Guide. Download the connector from the Cloudera Connectors page.   As always, we welcome your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external JIRA projects on issues.cloudera.org. ... View more

We are pleased to announce the release of Cloudera Enterprise 5.4.10 (CDH 5.4.10, Cloudera Manager 5.4.10, and Cloudera Navigator 2.3.10).   This release fixes key bugs and includes the following: CDH fixes for the following issues: FLUME-2712 - Optional channel errors slows down the Source to Main channel event rate FLUME-2886 - Optional Channels can cause OOMs HDFS-7553 - Fix the TestDFSUpgradeWithHA due to BindException HDFS-9083 - Replication violates block placement policy HDFS-9092 - Nfs silently drops overlapping write requests and causes data copying to fail HDFS-9406 - FSImage may get corrupted after deleting snapshot HDFS-9445 - Datanode may deadlock while handling a bad volume MAPREDUCE-6302 - Incorrect headroom can lead to a deadlock between map and reduce allocations MAPREDUCE-6460 - TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails YARN-2902 - Killing a container that is localizing can orphan resources in the DOWNLOADING state YARN-4204 - ConcurrentModificationException in FairSchedulerQueueInfo YARN-4347 - Resource manager fails with Null pointer exception HBASE-14621 - ReplicationLogCleaner stuck on RS crash HBASE-14923 - VerifyReplication should not mask the exception during result comparison HBASE-14926 - Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading HBASE-15019 - Replication stuck when HDFS is restarted HBASE-15031 - Fix merge of MVCC and SequenceID performance regression in branch-1.0 HBASE-15213 - Fix increment performance regression caused by HBASE-8763 on branch-1.0 HIVE-11826 - 'hadoop.proxyuser.hive.groups' configuration doesn't prevent unauthorized user to access metastore HIVE-12008 - Hive queries failing when using count(*) on column in view HIVE-12505 - Insert overwrite in same encrypted zone silently fails to remove some existing files HIVE-12566 - Incorrect result returns when using COALESCE in WHERE condition with LEFT JOIN HIVE-12784 - Group by SemanticException: Invalid column reference HIVE-12795 - Vectorized execution causes ClassCastException HIVE-13065 - Hive throws NPE when writing map type data to a HBase backed table IMPALA-1702 - Check for duplicate table IDs at the end of analysis (issue not entirely fixed, but now fails gracefully) IMPALA-2264 - Implicit casts to integers from decimals with higher precision sometimes allowed IMPALA-2473 - Excessive memory usage by scan nodes IMPALA-2643 - Nested inline view produces incorrect result when referencing the same column implicitly IMPALA-2765 - AnalysisException: operands of type BOOLEAN and TIMESTAMP are not comparable when OUTER JOIN with CASE statement IMPALA-2798 - After adding a column to avro table, Impala returns weird result if codegen is enabled. IMPALA-3034 - MemTracker leak on PHJ failure to spill IMPALA-3093 - ReopenClient() could NULL out 'client_key' causing a crash KITE-1114 - Kite CLI json-import HDFS temp file path not multiuser safe OOZIE-2413 - Kerberos credentials can expire if the KDC is slow to respond SENTRY-835 - Drop table leaves a connection open when using MetastoreListener SENTRY-953 - External Partitions which are referenced by more than one table can cause some unexpected behavior with Sentry HDFS sync SENTRY-957 - Exceptions in MetastoreCacheInitializer should probably not prevent HMS from starting up SENTRY-1002 - PathsUpdate.parsePath(path) will throw an NPE when parsing relative paths SENTRY-1044 - Tables with non-HDFS locations breaks HMS startup SOLR-8372 - backportCanceled recovery can lead to data loss SQOOP-2847 - Sqoop --incremental + missing parent --target-dir reports success with no data For a full list of upstream JIRAs fixed in CDH 5.4.10, see the issues fixed section of the Release Notes. Cloudera Manager fixes for the following issues: By default, hive.compute.query.using.stats was enabled. This produced incorrect results for some queries that used stats only. This setting is now disabled by default. After security was enabled on a cluster, YARN jobs failed. This occurred because the contents of any previously existing YARN User Cache directory could not be overridden after security was enabled. YARN jobs now complete as expected after security is enabled. Diagnostic bundles collected manually included all expected logs, but bundles collected on a schedule did not include role logs. Scheduled diagnostic bundles now include all expected logs. For full list of issues fixed in Cloudera Manager 5.4.10, see the issues fixed section of the Release Notes. We look forward to you trying it, using the information below: Download Cloudera Enterprise View the documentation   As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org. ... View more

We are pleased to announce the release of the Cloudera Distribution of Apache Kafka 2.0.1 for CDH 5.  Apache Kafka is a highly scalable, distributed, publish-subscribe messaging system. Apache Kafka 2.0.1 is maintenance release for Kafka 2.0 with bug fixes listed below. Notable Issues Fixed in Cloudera Distribution of Apache Kafka 2.0.1 Notable fixes backported into Kafka 2.0.1: KAFKA-3409: MirrorMaker hangs indefinitely due to commit. KAFKA-3378: Client blocks forever if SocketChannel connects instantly. KAFKA-3426: Improve protocol type errors when invalid sizes are received. KAFKA-3330: Truncate log cleaner offset checkpoint if the log is truncated. KAFKA-3463: Change default receive buffer size for consumer to 64K. KAFKA-1148: Delayed fetch/producer requests should be satisfied on a leader change. KAFKA-3352: Avoid DNS reverse lookups. KAFKA-3341: Improve error handling on invalid requests. KAFKA-3310: Fix for NPEs observed when throttling clients. KAFKA-2784: swallow exceptions when MirrorMaker exits. KAFKA-3243: Fix Kafka basic ops documentation for MirrorMaker, blacklist is not supported for new consumers. KAFKA-3235: Unclosed stream in AppInfoParser static block. KAFKA-3147: Memory records is not writable in MirrorMaker. KAFKA-3088: Broker crash on receipt of produce request with empty client ID. KAFKA-3159: Kafka consumer client poll is very CPU intensive under certain conditions. KAFKA-3189: Kafka server returns UnknownServerException for inherited exceptions. KAFKA-3157: MirrorMaker does not commit offset with low traffic. KAFKA-3179: Kafka consumer delivers message whose offset is earlier than sought offset. KAFKA-3198: Ticket Renewal Thread exits prematurely due to inverted comparison. All backported fixes can be viewed in the git release notes here.   We look forward to you trying Kafka 2.0.1! For more information, please use the links below: Install or upgrade Kafka Review the documentation Review the Release Notes As always, we welcome your feedback. Please send your comments and suggestions through our community forums. ... View more

We are pleased to announce the release of Hive ODBC v2.5.18 and Impala ODBC v2.5.32 drivers. This release has the following fixes and enhancements: Hive ODBC v2.5.18 Enhancements & New Features Updated handling of result set metadata When connected to Hive 0.14 or later, the driver now reports result set columns as being updatable, improving compatibility with the DotNet Odbc library and OdbcDataAdapter. Updated default socket timeout threshold The default value of the Socket Timeout option (the SocketTimeout key) has been changed from 30 to 60. Resolved Issues When attempting to connect to the server using a non-Windows version of the driver with Service Discovery Mode enabled, the client stops working When executing a parameterized INSERT statement on a DATE, DECIMAL, or TIMESTAMP column, an error occurs When executing a query that contains a large number of filters using OR operators, an error occurs   Impala ODBC v2.5.32 Advanced Kerberos authentication mechanism implemented The driver now supports Advanced Kerberos authentication, which allows concurrent connections within the same process to use different Kerberos user principals. For information about how to use this authentication mechanism, see the Cloudera ODBC Driver for Impala Installation and Configuration Guide. Updated handling of result set metadata The driver now reports result set columns as being updatable, improving compatibility with the DotNet Odbc library and OdbcDataAdapter. Known issues GetTables will return a remarks column, but the column will be empty even if there are remarks defined for tables (IMPALA-3193) Getting Started with the Cloudera Drivers Read the Cloudera ODBC 2.5.18 Driver for Hive Release Notes and Installation Guide. Read the Cloudera ODBC 2.5.32 Driver for Impala Release Notes and Installation Guide. Download the connector from the Cloudera Connectors page.   As always, we welcome your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external JIRA projects on issues.cloudera.org.   ... View more

The Kudu team is happy to announce the availability of Kudu 0.7.1. Kudu is currently undergoing Incubation at the Apache Software Foundation.   Kudu 0.7.1 is a bug fix release only for Kudu 0.7.0. Highlights: Notable bug fixes: KUDU-1325 fixes a tablet server crash that could occur during table deletion. In some cases, while a table was being deleted, other replicas would attempt to re-replicate tablets to servers that had already processed the deletion. This could trigger a race condition that caused a crash. KUDU-1341 fixes a potential data corruption and crash that could happen shortly after tablet server restarts in workloads that repeatedly delete and re-insert rows with the same primary key. In most cases, this corruption affected only a single replica and could be repaired by re-replicating from another. KUDU-1343 fixes a bug in the Java client that occurs when a scanner has to scan multiple batches from one tablet and then start scanning from another. In particular, this would affect any scans using the Java client that read large numbers of rows from multi-tablet tables. KUDU-1345 fixes a bug where in some cases the hybrid clock could jump backwards, resulting in a crash followed by an inability to restart the affected tablet server. KUDU-1360 fixes a bug in the kudu-spark module which prevented reading rows with NULL values. For a complete list of new features, changes, bug fixes, and known issues, see the Kudu 0.7.1 Release Notes . As always, your feedback is appreciated. For general Kudu questions, please visit the community page. If you have any questions related to Kudu in Cloudera context, please visit the Cloudera Community Forum. ... View more

The Kudu team is happy to announce the availability of Kudu 0.7.0. Kudu is currently undergoing Incubation at the Apache Software Foundation, with a beta available from Cloudera.   Kudu 0.7.0 includes improvements to client interfaces and metrics, and fixes for several key issues. We’re also releasing a refresh of the Impala Kudu parcel.   Highlights:   New features and improvements: The Python client has been completely rewritten, with a focus on improving code quality and testing. The read path (scanners) has been improved by adding many of the features already supported by the C++ and Java clients. The Python client is no longer considered experimental. KUDU-1321 Spark integration: a new API, kuduRDD,has been added, which wraps newAPIHadoopRDD and includes a default source for Spark SQL. KUDU-1250 The Java client includes new methods countPendingErrors() and getPendingErrors() on KuduSession. These methods allow you to count and retrieve outstanding row errors when configuring sessions with AUTO_FLUSH_BACKGROUND. New server-level metrics allow you to monitor CPU usage and context switching. KUDU-1317 The master now attempts to spread tablets more evenly across the cluster during table creation. This has no impact on existing tables, but will improve the speed at which under-replicated tablets are re-replicated after a tablet server failure.   Notable bug fixes: KUDU-1288 fixes a file descriptor leak, which could previously only be worked around by restarting the tablet server. KUDU-1250 fixes a hang in the Java client when processing an in-flight batch and the previous batch encountered an error.   Notable Improvements/bug fixes in the new Impala Kudu: Note: this release is still based on last summer’s pre-C5.5 fork of Impala and doesn’t have new features like nested types. IMPALA-2635 fixes a common bug seen when compute stats wasn’t run and the query contained a UNION. IMPALA-2740 fixes a bug reported on the mailing list where NULL values could be mishandled. KUDU-1184 is actually a fix on the Impala side, new Kudu tables created via Impala will now default to the Kudu master’s default replication value, instead of always being 1. A change was also made to leverage the new C++ Scan API introduced in KUDU-1259 which speeds up queries that handle small rows, especially row counts.   For a complete list of new features, changes, bug fixes, and known issues, see the Kudu 0.7.0 Release Notes . As always, your feedback is appreciated. For general Kudu questions, please visit the community page. If you have any questions related to Kudu in Cloudera context, please visit the Cloudera Community Forum. ... View more

We are pleased to announce the release of Cloudera Enterprise 5.6. This release introduces a pluggable storage interface to allow support and management of data on shared storage appliances instead of on traditional direct attached storage.  EMC's newly released all-flash storage appliance, the DSSD D5, has been fully tested and qualified with this release and is now supported for CDH and Cloudera Manager.   Cloudera Enterprise 5.6.0 is identical to Cloudera Enterprise 5.5.2 except for the support for shared storage as described above. If you do not need DSSD support, you do not need to upgrade if you are already using the latest 5.5.x release (currently 5.5.2).   We look forward to you trying it, using the information below: Download Cloudera Enterprise View the documentation   For more information about the DSSD D5, see this Cloudera VISION Blog post by our Chief Strategy Officer Mike Olson.   As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org. ... View more

On February 9th, 2016,  we announced the release of Cloudera Enterprise 5.5.2. Shortly after that, we discovered a severe bug in Cloudera Manager 5.5.2, so we decided to remove the release. CDH 5.5.2 was not affected by this bug.   Today we are releasing CDH 5.5.2, Cloudera Manager 5.5.2 and Cloudera Manager 5.5.3. All improvements, fixes, and changes in Cloudera Manager 5.5.2, as well as a change related to LDAP authentication, are included in Cloudera Manager 5.5.3. If you are upgrading from a previous version of Cloudera Manager, Cloudera recommends that you upgrade to version 5.5.3 or higher.     This release fixes key bugs and includes the following: CDH fixes for the following issues: (HBase) RegionCoprocessorHost System.nanoTime() performance bottleneck. (Hadoop) Exception may be swallowed in KMSClientProvider. (HDFS) Fix the TestDFSUpgradeWithHA due to BindException. (HDFS) Replication violates block placement policy. (HDFS) HttpFS does not parse ACL syntax correctly for operation REMOVEACLENTRIES. (HDFS) Possible NullPointerException in BlockManager if no excess replica can be chosen. (MapReduce) Multibyte delimiters with LineRecordReader cause duplicate records. (Yarn) Resource manager fails with NullPointerException. (HBase) VerifyReplication should not mask the exception during result comparison. (Hive) 'hadoop.proxyuser.hive.groups' configuration does not prevent an unauthorized user from accessing metastore. (Hive) Exception on HBaseSerDe.serialize binary field. (Impala) Throw AnalysisError if table properties are too large. (Impala) Re-enable SSL and Kerberos on server-server. (Oozie) Kerberos credentials can expire if the KDC is slow to respond. For a full list of upstream JIRAs fixed in CDH 5.5.2, see the issues fixed section of the Release Notes. Cloudera Manager 5.5.2 fixes for the following issues: Cross-site scripting vulnerability using malformed strings in the parcel remote URL list. In secure clusters, DataNode fails to start when dfs.data.transfer.protection is set and DataNode ports changed to unprivileged ports. Updating the Hive NameNode location multiple times could lead to data corruption. Cloudera Manager skips NameNode logs in the diagnostic bundle. Kafka 2.0 fails to deploy on large clusters because reserved.broker.max.id defaults to 1000. Spark REST API does not work when parcels are used. Update Apache Commons Collections library in Cloudera Manager due to major security vulnerability. Oozie keystore and truststore passwords not redacted.   For full list of issues fixed in Cloudera Manager 5.5.2, see the issues fixed section of the Release Notes. Cloudera Manager 5.5.3 fixes for the following issue: When upgrading to Cloudera Manager 5.5.2 from versions 5.4.8 or earlier, customers who have non-read-only roles configured through LDAP, and who have not explicitly set Cloudera Manager local roles, may lose their Cloudera Manager administration privileges.   All improvements, fixes, and changes in Cloudera Manager 5.5.2, as well as a change related to LDAP authentication, are included in Cloudera Manager 5.5.3. For more details, see the issues fixed section of the Release Notes. Cloudera Navigator 2.4.2 fixes for the following issues: After upgrade, entities are incorrectly marked as deleted. When Navigator uses the Cloudera Manager authenticator and Cloudera Manager is TLS enabled, authentication fails. Policies created from the Analytics UI were not working.   For a full list of issues fixed in Cloudera Navigator 2.4.2, see the issues fixed section of the Release Notes.   In the Cloudera Navigator 2.4.3 release, there are no new features or fixed issues. We look forward to you trying it, using the information below: Download Cloudera Enterprise View the documentation   As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external JIRA projects on issues.cloudera.org.   ... View more